History#

0.19.3 (2025-03-07)#

Internal Changes#

  • Use orjson instead of ujson due to the package not being maintained anymore. The developer of ujson raised conrcerns about security vulnerabilities and recommended using orjson instead.

  • Refactor the package to run in Jupyter notebooks without using nest_asyncio. This is done by creating and initializing a single global event loop thread dedicated to only running the asynchronous parts of this package.

  • Use threading.Event in _AsyncLoopThread for a more robust way to signal the event loop thread to stop running. This should prevent the event loop thread from hanging when the main thread exits before the event loop thread is done running. Overall, this improves thread safety and robustness of the package.

  • More robust handling of starting and stopping threads by lazy generation of a dedicated thread for the library and making _AsyncLoopThread a singleton. This can avoid issues that might arise from using TinyRetriever with other libraries that also use threads such as shapely.

0.19.1 (2025-01-17)#

Internal Changes#

  • Replace max_connection argument in all retrieve* functions with limit_per_host. This uses the aiohttp argument limit_per_host to limit the number of connections per host which prevents the server from being overwhelmed with too many connections. The default value is set to 5.

  • Add aiofiles as a new dependency for writing responses to a file in the stream_write function. This should speed up the writing process and make it more efficient.

Bug Fixes#

  • Fix an issue with running the library inside IPython terminal. (GH 50)

Breaking Changes#

  • Move stream_write function to its own module called streaming. This is to separate the streaming functionality from the main module and make it easier to maintain.

  • Simplify the stream_write function to only accept a list of URLs and a directory path to save the files. It no longer accepts passing keyword arguments and assumes that the user has added the necessary arguments to the URLs itself and is encoded correctly. Additionally, the chunk size by default is set to 1 MB.

0.18.0 (2024-10-05)#

Bug Fixes#

  • Bump the minimum version of aiohttp-client-cache>=0.12.3 to fix an issue with the latest version of aiohttp. (GH 48)

Breaking Changes#

  • Drop support for Python 3.8 since its end-of-life date is October 2024.

  • Remove all exceptions from the main module and raise them from the exceptions module. This is to declutter the public API and make it easier to maintain.

Internal Changes#

  • Make nest_asyncio a required dependency since most users will need it to run the package in Jupyter notebooks.

0.17.1 (2024-09-14)#

Internal Changes#

  • Drop support for Python 3.8 since its end-of-life date is October 2024.

0.17.0 (2024-05-07)#

Internal Changes#

  • Add the exceptions module to the high-level API to declutter the main module. In the future, all exceptions will be raised from this module and not from the main module. For now, the exceptions are raised from both modules for backward compatibility.

  • Switch to using the src layout instead of the flat layout for the package structure. This is to make the package more maintainable and to avoid any potential conflicts with other packages.

  • Add artifact attestations to the release workflow.

0.16.1 (2024-04-24)#

New Features#

  • When a request fails with encoding error, it gets retried with the encoding set to latine1. This is to handle cases where the server sends a response with an encoding that is not supported by the client. (GH 120)

0.16.0 (2024-01-03)#

New Features#

  • Add a new environmental variable called "HYRIVER_SSL_CERT" for setting the path to a SSL certificate file other than the default one. You can do this like so:

import os

os.environ["HYRIVER_SSL_CERT"] = "path/to/file.pem"

0.15.2 (2023-09-22)#

Bug Fixes#

  • Fix an issue with getting all valid keywords that aiohttp accepts by using aiohttp.ClientSession()._request directly.

0.15.0 (2023-05-07)#

From release 0.15 onward, all minor versions of HyRiver packages will be pinned. This ensures that previous minor versions of HyRiver packages cannot be installed with later minor releases. For example, if you have py3dep==0.14.x installed, you cannot install pydaymet==0.15.x. This is to ensure that the API is consistent across all minor versions.

Bug Fixes#

  • When raise_status is False, responses for failed requests used to return as None but their requests ID was not returned, so sorting would have failed. Now request IDs are returned for all requests regardless of whether they were successful or not.

  • Give precedence to non-default arguments for caching related arguments instead of directly getting them from env variables. This is to avoid the case where the user sets the env variables but then passes different arguments to the function. In this case, the function should use the passed arguments instead of the env variables.

0.14.0 (2023-03-05)#

New Features#

  • Add a new option to all functions called raise_status. If False no exception will be raised and instead None is returned for those requests that led to exceptions. This will allow for returning all responses that were successful and ignoring the ones that failed. This option defaults to True for retaining backward compatibility.

  • Set the cache expiration time to one week from never expire. To ensure all users have a smooth transition, cache files that were created before the release of this version will be deleted, and a new cache will be created.

Internal Changes#

  • Sync all minor versions of HyRiver packages to 0.14.0.

0.3.12 (2023-02-10)#

Internal Changes#

  • Rewrite the private async_session function as two separate functions called async_session_without_cache and async_session_with_cache. This makes the code more readable and easier to maintain.

  • Fully migrate setup.cfg and setup.py to pyproject.toml.

  • Convert relative imports to absolute with absolufy-imports.

  • Make utils module private.

  • Sync all patch versions of HyRiver packages to x.x.12.

0.3.10 (2023-01-08)#

New Features#

  • Refactor the show_versions function to improve performance and print the output in a nicer table-like format.

Bug Fixes#

  • Fix a bug in reading the HYRIVER_CACHE_EXPIRE environmental variable.

  • Bump the minimum version of aiohttp-client-cache to 0.8.1 to fix a bug in reading cache files that were created with previous versions. (GH 41)

Internal Changes#

  • Enable fast_save in aiohttp-client-cache to speed up saving responses to the cache file.

  • Use pyright for type checking instead of mypy and fix all type errors.

  • Skip 0.13.8/9 versions so the minor version of all HyRiver packages become the same.

0.3.7 (2022-12-09)#

New Features#

  • Add support for specifying the chunk size in stream_write. Defaults to None which was the default behavior before, and means iterating over and writing the responses as they are received from the server.

Internal Changes#

  • Use pyupgrade package to update the type hinting annotations to Python 3.10 style.

  • Modify the codebase based on Refurb suggestions.

0.3.6 (2022-08-30)#

Internal Changes#

  • Add the missing PyPi classifiers for the supported Python versions.

  • Release the package as both async_retriever and async-retriever on PyPi and Conda-forge.

0.3.5 (2022-08-29)#

Breaking Changes#

  • Append “Error” to all exception classes for conforming to PEP-8 naming conventions.

Internal Changes#

  • Bump minimum version of aiohttp-client-cache to 0.7.3 since the attrs version issue has been addressed.

0.3.4 (2022-07-31)#

New Features#

  • Add a new function, stream_write, for writing a response to a file as it’s being retrieved. This could be very useful for downloading large files. This function does not use persistent caching.

0.3.3 (2022-06-14)#

Breaking Changes#

  • Set the minimum supported version of Python to 3.8 since many of the dependencies such as xarray, pandas, rioxarray have dropped support for Python 3.7.

Internal Changes#

  • Use micromamba for running tests and use nox for linting in CI.

0.3.2 (2022-04-03)#

New Features#

  • Add support for setting caching-related arguments using three environmental variables:

    • HYRIVER_CACHE_NAME: Path to the caching SQLite database.

    • HYRIVER_CACHE_EXPIRE: Expiration time for cached requests in seconds.

    • HYRIVER_CACHE_DISABLE: Disable reading/writing from/to the cache file.

    You can do this like so:

import os

os.environ["HYRIVER_CACHE_NAME"] = "path/to/file.sqlite"
os.environ["HYRIVER_CACHE_EXPIRE"] = "3600"
os.environ["HYRIVER_CACHE_DISABLE"] = "true"

Internal Changes#

  • Include the URL of a failed request in its exception error message.

0.3.1 (2021-12-31)#

New Features#

  • Add three new functions called retrieve_text, retrieve_json, and retrieve_binary. These functions are derived from the retrieve function and are used to retrieve the text, JSON, or binary content of a response. They are meant to help with type hinting since they have only one return type instead of the three different return types that the retrieve function has.

Internal Changes#

  • Move all private functions to a new module called utils. This makes the code-base more readable and easier to maintain.

0.3.0 (2021-12-27)#

Breaking Changes#

  • Set the expiration time to never expire by default.

New Features#

  • Add two new arguments to retrieve for controlling caching. First, delete_url_cache for deleting caches for specific requests. Second, expire_after for setting a custom expiration time.

  • Expose the ssl argument for disabling the SSL certification verification (GH 41).

  • Add a new option called disable that temporarily disables caching requests/responses if set to True. It defaults to False.

0.2.5 (2021-11-09)#

New Features#

  • Add two new arguments, timeout and expire_after, to retrieve. These two arguments give the user more control in dealing with issues related to caching.

Internal Changes#

  • Revert to pytest as the testing framework.

  • Use importlib-metadata for getting the version instead of pkg_resources to decrease import time as discussed in this issue.

0.2.4 (2021-09-10)#

Internal Changes#

  • Use ujon for converting responses to JSON.

Bug Fixes#

  • Fix an issue with catching service error messages.

0.2.3 (2021-08-26)#

Internal Changes#

  • Use ujson for JSON parsing instead of orjson since orjson only serializes to bytes which is not compatible with aiohttp.

0.2.2 (2021-08-19)#

New Features#

  • Add a new function, clean_cache, for manually removing the expired responses from the cache database.

Internal Changes#

  • Handle all cache file-related operations in the create_cachefile function.

0.2.1 (2021-07-31)#

New Features#

  • The responses now are returned to the same order as the input URLs.

  • Add support for passing connection type, i.e., IPv4 only, IPv6 only, or both via the family argument. Defaults to both.

  • Set trust_env=True, so the session can read the system’s netrc files. This can be useful for working with services such as EarthData service that read the user authentication info from a netrc file.

Internal Changes#

  • Replace the AsyncRequest class with the _retrieve function to increase readability and reduce overhead.

  • More robust handling of validating user inputs via a new class called ValidateInputs.

  • Move all if-blocks in async_session to other functions to improve performance.

0.2.0 (2021-06-17)#

Breaking Changes#

  • Make persistent caching dependencies required.

  • Rename request argument to request_method in retrieve which now accepts both lower and upper cases of get and post.

Bug Fixes#

  • Pass a new loop explicitly to nest_asyncio (GH 1).

Internal Changes#

  • Refactor the entire code-base for more efficient handling of different request methods.

  • Check the validity of inputs before sending requests.

  • Improve documentation.

  • Improve cache handling by removing the expired responses before returning the results.

  • Increase testing coverage to 100%.

0.1.0 (2021-05-01)#

  • Initial release.