Constructing Queries

[TelloViz]

Constructing Queries

Our program needed a way to transform the front end filters and parameters into a valid query string that the API could recognize.

My Solution

1. The front end uses a signal/slot mechanism to pass a

list[tuple[str, str, str]]

to the function

BackendController.request_search(list[tuple[str, str, str]])

Example list of tuples:

[(, name, Rayquaza), (set, name, Roaring Skies), (types, , colorless)]

2. BackendController.request_search then passes the unaltered list to

SearchHandler.handle_request_search(list[tuple[str,str,str]])

3. SearchHandler.handle_request_search then passes the parameter to

Backend.construct_query(list[tuple[str, str, str]])

4. construct_query returns the query as a string that the API recognizes as a valid query.

Example Query String:

"(name:"Rayquaza") AND (set.name:"Roaring Skies") AND (types:"colorless")"

5. SearchHandler.handle_request_search then uses this constructed query to call the function:

Backend.query_api(str)

6. From here the retrieved data is passed back up the chain, but this is outside of the scope of constructing queries.

---

construct_query Implementation

# test_backend.py
# Created by Josh Lollis
# 10/19/2024

def construct_query(self, query_data: list[tuple[str, str, str]]) -> str:
        """
        Constructs a query string from the given search parameters.
        Args:
            search_parameters (list[tuple[str,str,str]]): A list of tuples representing the search parameters.

        Returns:
            str: The constructed query string.
        """

        # Initialize an empty dictionary to hold query parts
        query_parts = {}

        # Loop through each item in the queryData list
        for item in query_data:
            # Extract the category, subcategory, and target from the item
            category = item[0]
            subcategory = item[1]
            target = item[2]

            # Prepare the target for the query string with quotes
            target = f'"{target}"'  # Enclose target in quotes

            # If the category is not already in query_parts, create a new list for it
            if category not in query_parts:
                query_parts[category] = []

            # Check if subcategory is provided
            if subcategory:
                if category:
                    # Create a string in the format "category.subcategory:target" 
                    # and add it to the list
                    query_parts[category].append(f"{category}.{subcategory}:{target}")
                else:
                    # If no category, just "subcategory:target"
                    query_parts[category].append(f"{subcategory}:{target}")
            else:
                if category:
                    # Create a string in the format "category:target" 
                    # and add it to the list
                    query_parts[category].append(f"{category}:{target}")
                else:
                    # If no category or subcategory, just "target"
                    query_parts[category].append(target)

        # Prepare to construct the final query
        final_query_parts = []

        # Loop through each list of items in query_parts
        for items in query_parts.values():
            # If there is more than one item in the list
            if len(items) > 1:
                # Combine the items with "OR" and add parentheses around them
                final_query_parts.append(f"({' OR '.join(items)})")
            else:
                # If there's only one item, still wrap it in parentheses
                final_query_parts.append(f"({items[0]})")

        # Join all parts with "AND" to form the final query string
        final_query = " AND ".join(final_query_parts)
        return final_query

---

Testing construct_query

# test_backend.py
# Created by Josh Lollis
# 10/19/2024

import time
from backend import Backend
from pokemontcgsdk import Card 

def test_construct_query():
    """
    Summary:
        Passes a set of test search data to Backend.construct_query, 
        then queries the api using them to test their validity. 
        Then the function returns the result count for each query, 
        and averages the response time per result.
    Args:
        None
    Return:
        None
    """

    backend = Backend()

    # Test data for testing the construct_query function
    test_data_sets = [
        [
            ('name', '', 'Pikachu'),
        ],
        [
            ('set', 'name', 'Base'),
            ('types', '', 'Electric'),
            ('types', '', 'Grass'),
        ],
        [
            ('set', 'name', 'Jungle'),
            ('', 'name', 'Jigglypuff'),
        ],
        [
            ('set', 'name', 'Team Rocket'),
            ('', 'name', 'Charmander'),
            ('types', '', 'Fire'),
            ('types', '', 'Dark'),
            ('types', '', 'Dragon'),
        ],
        [
            ('set', 'name', 'Neo Genesis'),
            ('set', 'name', 'Neo Discovery'),
            ('', 'name', 'Lugia'),
        ],
        [
            ('set', 'name', 'Aquapolis'),
            ('types', '', 'Water'),
            ('types', '', 'Fire'),
            ('types', '', 'Psychic'),
            ('types', '', 'Dragon'),
            ('types', '', 'Fairy'),
        ],
        [
            ('set', 'name', 'Fossil'),
            ('set', 'name', 'Base Set 2'),
            ('set', 'name', 'Skyridge'),
            ('', 'name', 'Mewtwo'),
        ],
        [
            ('name', '', 'Bulbasaur'),
            ('types', '', 'Grass'),
            ('types', '', 'Poison'),
            ('types', '', 'Psychic'),
            ('types', '', 'Fairy'),
        ],
        [
            ('set', 'name', 'Hidden Legends'),
            ('set', 'name', 'FireRed & LeafGreen'),
            ('set', 'name', 'Unseen Forces'),
            ('set', 'name', 'EX Trainer Kit Latios'),
        ],
        [
            ('set', 'name', 'Legendary Collection'),
            ('', 'name', 'Gengar'),
            ('types', '', 'Ghost'),
            ('types', '', 'Poison'),
            ('types', '', 'Psychic'),
            ('types', '', 'Dark'),
            ('types', '', 'Fairy'),
            ('types', '', 'Dragon'),
            ('types', '', 'Steel'),
        ],
        [
            ('name', '', 'Eevee'),
            ('types', '', 'Normal'),
            ('types', '', 'Fire'),
            ('types', '', 'Water'),
            ('types', '', 'Electric'),
            ('types', '', 'Psychic'),
            ('types', '', 'Dark'),
        ],
        [
            ('set', 'name', 'XY'),
            ('set', 'name', 'Hidden Fates'),
            ('types', '', 'Grass'),
            ('types', '', 'Fire'),
            ('types', '', 'Water'),
            ('types', '', 'Electric'),
            ('types', '', 'Psychic'),
            ('types', '', 'Dark'),
            ('types', '', 'Dragon'),
            ('types', '', 'Steel'),
            ('types', '', 'Fairy'),
            ('types', '', 'Fighting'),
            ('types', '', 'Colorless'),
        ],
        [
            ('set', 'name', 'Sun & Moon'),
            ('', 'name', 'Rowlet'),
        ],
        [
            ('set', 'name', 'Primal Clash'),
            ('set', 'name', 'Double Crisis'),
            ('set', 'name', 'Roaring Skies'),
            ('set', 'name', 'Ancient Origins'),
            ('set', 'name', 'BREAKthrough'),
        ],
        [
            ('set', 'name', "Champion's Path"),
            ('types', '', 'Grass'),
            ('types', '', 'Fire'),
            ('types', '', 'Water'),
            ('types', '', 'Electric'),
            ('types', '', 'Psychic'),
            ('types', '', 'Dark'),
            ('types', '', 'Dragon'),
            ('types', '', 'Steel'),
            ('types', '', 'Fairy'),
        ],
        [
            ('set', 'name', 'Cosmic Eclipse'),
            ('set', 'name', 'Unbroken Bonds'),
            ('', 'name', 'Charizard'),
            ('types', '', 'Fire'),
            ('types', '', 'Flying'),
            ('types', '', 'Dragon'),
        ],
        [
            ('name', '', 'Machamp'),
            ('types', '', 'Fighting'),
            ('types', '', 'Dark'),
        ],
        [
            ('set', 'name', 'Evolving Skies'),
            ('set', 'name', 'Fusion Strike'),
            ('set', 'name', 'Brilliant Stars'),
            ('types', '', 'Grass'),
            ('types', '', 'Fire'),
            ('types', '', 'Water'),
            ('types', '', 'Electric'),
            ('types', '', 'Dark'),
            ('types', '', 'Dragon'),
        ],
        [
            ('set', 'name', 'Astral Radiance'),
            ('set', 'name', 'Pokémon GO'),
            ('set', 'name', 'Lost Origin'),
            ('set', 'name', 'Silver Tempest'),
            ('set', 'name', 'Obsidian Flames'),
            ('', 'name', 'Snorlax'),
        ]
    ]

    total_time = 0
    total_results = 0

    # Process each test case
    for test_data in test_data_sets:
        # Skip overly broad filters (e.g., ones that just look for all types)
        if any(x[0] == 'types' and x[1] == '' for x in test_data):
            continue

        # Construct query
        test_query = backend.construct_query(test_data)

        # Start timing the API request
        start_time = time.time()

        # Query the API
        results = Card.where(q=test_query)

        # Measure the time taken
        elapsed_time = time.time() - start_time

        # Update total time and results
        total_time += elapsed_time
        total_results += len(results)

        # Output the results for this query
        print(f"Input Data: {test_data}")
        print(f"Constructed Query: {test_query}")
        print(f"Number of results: {len(results)}")
        print(f"Time taken: {elapsed_time:.2f} seconds\n")

    # Calculate average response time per result
    if total_results > 0:
        average_response_time = total_time / total_results
        print(f"Total time for all queries: {total_time:.2f} seconds")
        print(f"Total number of results: {total_results}")
        print(f"Average response time per result: {average_response_time:.4f} seconds\n")
    else:
        print("No results returned from any queries.")

if __name__ == "__main__":
    test_construct_query()
    ```

---

## Test Output

```bash
~/Desktop/cc/Card-Collection/CardCollection$ python3
Python 3.9.7 (default, Sep 29 2024, 14:13:07) 
[GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from test_backend import test_construct_query
>>> test_construct_query()
Input Data: [('name', '', 'Pikachu')]
Constructed Query: (name:"Pikachu")
Number of results: 167
Time taken: 5.84 seconds

Input Data: [('set', 'name', 'Jungle'), ('', 'name', 'Jigglypuff')]
Constructed Query: (set.name:"Jungle") AND (name:"Jigglypuff")
Number of results: 1
Time taken: 1.23 seconds

Input Data: [('set', 'name', 'Neo Genesis'), ('set', 'name', 'Neo Discovery'), ('', 'name', 'Lugia')]
Constructed Query: (set.name:"Neo Genesis" OR set.name:"Neo Discovery") AND (name:"Lugia")
Number of results: 1
Time taken: 1.12 seconds

Input Data: [('set', 'name', 'Fossil'), ('set', 'name', 'Base Set 2'), ('set', 'name', 'Skyridge'), ('', 'name', 'Mewtwo')]
Constructed Query: (set.name:"Fossil" OR set.name:"Base Set 2" OR set.name:"Skyridge") AND (name:"Mewtwo")
Number of results: 1
Time taken: 1.13 seconds

Input Data: [('set', 'name', 'Hidden Legends'), ('set', 'name', 'FireRed & LeafGreen'), ('set', 'name', 'Unseen Forces'), ('set', 'name', 'EX Trainer Kit Latios')]
Constructed Query: (set.name:"Hidden Legends" OR set.name:"FireRed & LeafGreen" OR set.name:"Unseen Forces" OR set.name:"EX Trainer Kit Latios")
Number of results: 373
Time taken: 11.88 seconds

Input Data: [('set', 'name', 'Sun & Moon'), ('', 'name', 'Rowlet')]
Constructed Query: (set.name:"Sun & Moon") AND (name:"Rowlet")
Number of results: 1
Time taken: 1.43 seconds

Input Data: [('set', 'name', 'Primal Clash'), ('set', 'name', 'Double Crisis'), ('set', 'name', 'Roaring Skies'), ('set', 'name', 'Ancient Origins'), ('set', 'name', 'BREAKthrough')]
Constructed Query: (set.name:"Primal Clash" OR set.name:"Double Crisis" OR set.name:"Roaring Skies" OR set.name:"Ancient Origins" OR set.name:"BREAKthrough")
Number of results: 576
Time taken: 18.12 seconds

Input Data: [('set', 'name', 'Astral Radiance'), ('set', 'name', 'Pokémon GO'), ('set', 'name', 'Lost Origin'), ('set', 'name', 'Silver Tempest'), ('set', 'name', 'Obsidian Flames'), ('', 'name', 'Snorlax')]
Constructed Query: (set.name:"Astral Radiance" OR set.name:"Pokémon GO" OR set.name:"Lost Origin" OR set.name:"Silver Tempest" OR set.name:"Obsidian Flames") AND (name:"Snorlax")
Number of results: 3
Time taken: 1.44 seconds

Total time for all queries: 42.19 seconds
Total number of results: 1123
Average response time per result: 0.0376 seconds