From release 0.15 onward, all minor versions of HyRiver packages
will be pinned. This ensures that previous minor versions of HyRiver
packages cannot be installed with later minor releases. For example,
if you have
pygeoogc==0.14.x installed, you cannot install
pygeoogc==0.15.x series. This is to ensure that the API is
consistent across all minor versions.
For now, retain compatibility with
Fix an issue in
getfeature_bygeommethod fails if the requested web service does not have
geometry_columnattribute in its schema. This release addresses this issue by trying to find the name from other attributes in the schema. If it fails to find, it raises a
Catch an edge case in
match_crsfunction where the input is a list of coordinates of length 4.
Give precedence to non-default arguments for caching related arguments instead of directly getting them from env variables. This is to avoid the case where the user sets the env variables but then passes different arguments to the function. In this case, the function should use the passed arguments instead of the env variables.
pyyamlas a dependency since it is not used anymore.
Bump the minimum required version of
shapelyto 2.0, and use its new API.
Sync all minor versions of HyRiver packages to 0.14.0.
match_crsless strict in terms of the input geometry type being
listby relying on
contextlib.suppress. So, now users can pass any combination of
tupleas coordinates or bounding box.
More robust handling of inputs and outputs in
streaming_download. Now, only if input is
strthe function returns a single
Pathobject. Previously if there was only one URL, whether
listof length one or
str, the output was a single
Path, which could have had unintended consequences.
WFSwhen some layers have missing schema info, the class failed to initialize. This release fixes this issue by ignoring layers with missing schema info and asks the user to pass a sort parameter instead of trying to automatically find a sort parameter. This fix also improves the performance of this function by making fewer web requests.
Convert relative imports to absolute with
Sync all patch versions of HyRiver packages to x.x.12.
pyrightfor type checking instead of
mypysince it is faster and more accurate. Also, fix all the type errors reported by
Improve code quality by addressing issues raised by DeepSource.
Add the missing annotation import to the
cache_keysto ensure Python 3.8 and 3.9 work with Python 3.10 style type hinting.
Add a new property to
schemathat contains information about column names and their types for all layers. It also the geometry type and its name for each layer.
Automatically determine the geometry keyword that should be passed to
WFS.getfeature_bygeomusing the new
Add support for disabling SSL verification to
Add support for streaming responses to
Add support for closing the session to
Add support for passing
postmethods. Previously, keyword
payloadwas used for
datacan also be passed as keyword arguments to these methods.
Add a new function called
streaming_downloadfor downloading large files in parallel and in chunks.
Fix an issue in
WFSclass where number of requested features exceeds the maximum number of features allowed by the server, but only a portion of the features are returned. This release addresses this issue by first getting only the number of features and then requesting the features in chunks of features IDs based on the maximum number of features allowed by the server.
Drop support for WFS version 1.0.0 since it does not support paging.
Modify the codebase based on Refurb suggestions.
Fix the warning message in
ArcGISRESTFulwhere wrong number of missing feature IDs were being reported.
Add a new method to
RetrySessionfor getting the request head called
RetrySession.head. This is useful for getting the headers of a request without having to make a full request which is useful for getting the
Content-Lengthheader for example, i.e., download size.
Fix an issue in the decompose function,
utils.bbox_decompose, where the generated bounding boxes might overlap in some cases. A new approach has been implemented based on finding the number of required bounding boxes from max allowable no. of pixels and total requested pixels without changing the input bounding box projection. This ensures that the decomposed bounding boxes are not overlapping so
xarray.open_mfdatasetcan be used without any issues.
utils.match_crsfunction, don’t perform any projection if the source target CRS are the same.
Improve type hints for CRS-related arguments of all functions by including string, integer, and
Add a new class method to
get_service_optionsfor retrieving the available layers, output formats, and CRSs for a given service. Here’s an example:
pyupgradepackage to update the type hinting annotations to Python 3.10 style.
from pygeoogc.core import WMSBase url = "https://elevation.nationalmap.gov/arcgis/services/3DEPElevation/ImageServer/WMSServer" wms = WMSBase(url, validation=False) wms.get_service_options() print(wms.available_layer)
Add the missing PyPi classifiers for the supported Python versions.
Append “Error” to all exception classes for conforming to PEP-8 naming conventions.
Bump minimum version of
owslibto 0.27.2 since the
pyprojincompatibility issue has been addressed in this issue.
Bump minimum version of
requests-cacheto 0.9.6 since the
attrsversion issue has been addressed.
Add support for disabling persistent caching in
RetrySessionvia an argument and also
Set the minimum supported version of Python to 3.8 since many of the dependencies such as
rioxarrayhave dropped support for Python 3.7.
owslibto version <0.26 since version 0.26 has pinned
pyprojto version <3.3 which is not compatible with
Use micromamba for running tests and use nox for linting in CI.
More robust handling of errors in
Noneresponses. Also, use the
ArcGISRESTful.bysqlsince the SQL Clause could be a long string.
Remove caching-related arguments from all functions since now they can be set globally via three environmental variables:
HYRIVER_CACHE_NAME: Path to the caching SQLite database.
HYRIVER_CACHE_EXPIRE: Expiration time for cached requests in seconds.
HYRIVER_CACHE_DISABLE: Disable reading/writing from/to the cache file.
You can do this like so:
import os os.environ["HYRIVER_CACHE_NAME"] = "path/to/file.sqlite" os.environ["HYRIVER_CACHE_EXPIRE"] = "3600" os.environ["HYRIVER_CACHE_DISABLE"] = "true"
ArcGISRESTful.oids_byfieldconvert the input
listif a user passes a single
ServicURLto hard code the supported links instead of reading them from a file. Also, the class now is based on
NamedTuplethat has a nicer
validate_crspublic that can be accessed from the
utilsmodule. This is useful for checking validity of user input CRS values and getting its string representation.
pygeoogc.utils.valid_wms_crsfunction for getting a list of valid CRS values from a WMS service.
Add 3DEP’s index WFS service for querying availability of 3DEP data within a bounding box.
Add type checking with
typeguardand fixed typing issues raised by
show_versionsto ensure getting correct versions of all dependencies.
Use the three new
ar.retrieve_*functions instead of the old
ar.retrievefunction to improve type hinting and to make the API more consistent.
Add a new argument to
verboseto turn on/off all info level logs.
Add an option to
get_geometryto turn on/off requesting the data with or without geometry.
ArcGISRESTfulsaves the object IDs of the features that user requested but are not available in the database to
Add a new parameter to
Truein case there are any failed queries, no retrying attempts is done and object IDs of the failed requests are saved to a text file which its path can be accessed via
Set response caching expiration time to never expire, for all base classes. A new argument has been added to all three base classes called
expire_afterthat can be used to set the expiration time.
Add a new method to all three base classes called
clear_cachethat clears all cached responses for that specific client.
ArcGISRESTfulclass now return a list of object IDs rather than setting
self.featureids. This makes it possible to pass the outputs of the
oids_by*functions directly to the
ArcGISRESTfulless cluttered by instantiating
ArcGISRESTfulrather than inheriting from its base class.
Explicitly set a minimum value of 1 for the maximum number of feature IDs per request in
Add all the missing types so
RetrySessionsince it can be easily be achieved using
with unittest.mock.patch("socket.has_ipv6", False):.
geomsmethod for iterating over geometries to address the deprecation warning of
importlib-metadatafor getting the version instead of
pkg_resourcesto decrease import time as discussed in this issue.
Remove unnecessary dependency on
Update the code to use the latest
Add URL for PyGeoAPI service.
Fix a bug in
WFS.getfeature_byidwhen the number of IDs exceeds the service’s limit by splitting large requests into multiple smaller requests.
Add two new arguments,
WFSto control the maximum number of records per request (defaults to 1000) and specify the response read method (defaults to
Simplify the retry logic
ArcGISRESTFulby making it run four times and making sure that the last retry is one object ID per request.
The highlight of this release is migrating to use
AsyncRetriever that can improve
the network response time significantly. Another highlight is a major refactoring of
ArcGISRESTFul that improves performance and reduce code complexity.
Add a new method to
ArcGISRESTFulclass for automatically retrying the failed requests. This private method plucks out individual features that were in a failed request with several features. This happens when there are some object IDs that are not available on the server, and they are included in the request. In these situations the request will fail, although there are valid object IDs in the request. This method will pluck out the valid object IDs.
Add support for passing additional parameters to
WMSrequests such as
Add support for WFS version 1.0.0.
requests-cachefor all the web services.
ServiceErrorSince it’s more representative of the intended exception.
Raise for response status in
RetrySessionbefore the try-except block so
RequestsExceptioncan raise, and its error messaged be parsed.
utils.threadingsince all threading operations are now handled by
Increase test coverage.
Add support for requesting
Add a new argument called
ArcGISRESTful.oids_bygeomfor specifying the buffer distance from the input geometry for getting features.
Drop support for Python 3.6 since many of the dependencies such as
pandashave done so.
async_requestsfunction, since it has been packaged as a new Python library called AsyncRetriever.
MatchCRS. Now, it should be instantiated by providing the in and out CRSs like so:
MatchCRS(in_crs, out_crs). Then its methods, namely,
coords, can be called. These methods now have only one input, geometry.
Change input and output types of
MatchCRS.coordsfrom tuple of lists of coordinates to list of
ArcGISRESTfulnow has a new argument,
layer, for specifying the layer number (int). Now, the target layer should either be a part of
base_urlor be passed with
oids_bygeommethod, since that’s where it’s applicable.
ArcGISRESTfulBaseclass to reduce its code complexity and make the service initialization logic much simpler. The class is faster since it makes fewer requests during the initialization process.
pydanticas a new dependency that takes care of
Use persistent caching for all send/receive requests that can significantly improve the network response time.
Explicitly include all the hard dependencies in
Set a default value of 1000 for
WFSBasesince support for Python 3.6 is dropped.
Add announcement regarding the new name for the software stack, HyRiver.
pipinstallation and release workflow.
The first release after renaming
ArcGISRESTfulbeing set to
feature typesproperty to
ArcGISRESTFulfor getting names and IDs of types of features in the database.
ElementTreesince it’s been deprecated by
Remove dependency on
dataclassessince its benefits and usage in the code was minimal.
Speed up CI testing by using
ArcGISRESTFullnow prints number of found features before attempting to retrieve them.
loggingmodule for printing information.
Bump version to the same version as PyGeoHydro.
Add support for query by point and multi-points to
Add support for buffer distance to
Add support for generating ESRI-based queries for points and multi-points to
Add all the missing type annotations.
Update the Daymet URL to version 4. You can check the release information here
cytoolzlibrary for improving performance of some operations.
ArcGISRESTfulclass that get the spatial extent of the service.
Add URL to
airmapservice for getting elevation data at 30 m resolution.
urlib3deprecation warning about using
Remove unused variables in
async_requestsissue on Windows systems.
Added/Renamed three class methods in
oids_bysql. So you can query feature within a geometry, using specific field ID(s), or more generally using any valid SQL 92 WHERE clause.
Added support for query with SQL WHERE clause to
Changed the NLDI’s URL for migrating to its new API v3.
Added support for CQL filter to
WFS, credits to Emilio.
Moved all the web services URLs to a YAML file that
ServiceURLclass reads. It makes managing the new URLs easier. The file is located at
Turned off threading by default for all the services since not all web services supports it.
Added support for setting the request method,
WFS.byfilter, which could be useful when the filter string is long.
Added support for asynchronous download via the function
bbox_decomposeto fix the
WMSissue with high resolution requests.
orjsonto speed up JSON operations.
Removed threading for
WMSdue to inconsistent behavior.
Addressed an issue with domain decomposition for
WMSwhere width/height becomes 0.
bbox_decompose. The function now decomposes the domain in both directions and return squares and rectangular.
wms_byboxfunction as a class called
WMSwith a similar interface to the
Added support for WMS 1.3.0 and WFS 2.0.0.
Added a custom
Exceptionfor the threading function called
WFSwhich is False by default. It is useful for cases where a web service doesn’t change the axis order from the transitional
yxfor versions higher than 1.3.0.
Remove unnecessary transformation of the input bbox in WFS.
Add the missing
max_pixelargument to the
onlyipv4to conform to the