Utilities

The LogQS client provides various utility functions to help users interact with LogQS resources more easily. A list of some of the available utilities is provided in the API Reference (see the Utils section of the API reference).

Listing All Resources

LogQS enforces a maximum limit (or page size) when listing resources. This means when listing a resource, you may not get all results in a single call. By default, list responses include a count field that indicates the total number of resources available given the parameters provided which can be used to determine if multiple calls are needed to retrieve all resources.

logs_list_response = lqs.list.log()
print(f"Retrieved {len(logs_list_response.data)} logs out of {logs_list_response.count} total logs.")

Typically, a user would need to paginate through the results using the limit and offset parameters to retrieve all resources, e.g.,

all_logs = []
limit = 100
offset = 0

while True:
    logs_list_response = lqs.list.log(limit=limit, offset=offset)
    all_logs.extend(logs_list_response.data)

    if len(all_logs) >= logs_list_response.count:
        break

    offset += limit

print(f"Retrieved a total of {len(all_logs)} logs.")

However, the LogQS client provides a utility method list_all() that abstracts away this pagination logic and retrieves all resources for you in a performant, multi-threaded manner:

all_logs = lqs.utils.list_all(lqs.list.log)
print(f"Retrieved a total of {len(all_logs)} logs.")

This method takes a Callable list_method as its first parameter (in this case, lqs.list.log) along with any parameters that should be passed to that method. It will then handle the pagination and return a list of all resources.

For example, to list all topics with a type name like Image:

all_image_topics = lqs.utils.list_all(
    lqs.list.topic,
    type_name="Image"
)
print(f"Retrieved a total of {len(all_image_topics)} Image topics.")