API reference

Functions and classes for running GCAM output database queries.

This module provides the public API for gcamreader: a Query container, a parse_batch_query() helper, local and remote database connection classes (LocalDBConn and RemoteDBConn), and the importdata() convenience function. Query results are returned as pandas DataFrames.

class gcamreader.querymi.Query(xmlin)[source]

Bases: object

A parsed GCAM query definition.

Parameters:

xmlin (str | ET._Element)

querystr

String representation of the query XML.

regions

List of region names parsed from the query, or None when the query does not specify regions.

title

The query title.

Examples

Build a query directly from an XML string:

>>> import gcamreader
>>> xml = (
...     '<supplyDemandQuery title="CO2 emissions">'
...     '<region name="USA"/>'
...     '</supplyDemandQuery>'
... )
>>> query = gcamreader.Query(xml)
>>> query.title
'CO2 emissions'
>>> query.regions
['USA']
gcamreader.querymi.parse_batch_query(filename)[source]

Parse a GCAM query file into a list of Query objects.

Parameters:

filename (str) – Path to a GCAM batch query XML file.

Return type:

list[Query]

Returns:

A list of Query objects, one per title element found.

Examples

Parse the sample land-allocation query bundled with the package:

>>> import os
>>> import gcamreader
>>> query_file = os.path.join(
...     gcamreader.sample_data_dir(),
...     "queries",
...     "query_land_reg32_basin235_gcam5p0.xml",
... )
>>> queries = gcamreader.parse_batch_query(query_file)
>>> queries[0].title
'Crop Land Allocation'
gcamreader.querymi.sample_data_dir()[source]

Return the filesystem path to the bundled sample data directory.

The sample data directory contains a small example BaseX database, an example query, and a reference output, all used by the test suite and examples.

Return type:

str

Returns:

The absolute path to the data package data directory.

Examples

>>> import os
>>> import gcamreader
>>> data_dir = gcamreader.sample_data_dir()
>>> os.path.isdir(data_dir)
True
class gcamreader.querymi.LocalDBConn(dbpath, dbfile, suppress_gabble=True, miclasspath=None, validatedb=True, maxMemory='4g')[source]

Bases: object

Connection to a local GCAM database.

A local database connection comprises a name and location for the database, along with a class path for the Java program used to extract data and some options passed to the functions that run the queries.

Parameters:

  • dbpath (str)

  • dbfile (str)

  • suppress_gabble (bool)

  • miclasspath (str | None)

  • validatedb (bool)

  • maxMemory (str)

dbpath

Absolute path to the directory containing the GCAM database.

dbfile

Name of the GCAM database.

suppress_gabble

Whether to suppress model interface console output.

maxMemory

Maximum memory passed to the Java runtime.

miclasspath

Java class path for the GCAM model interface.

Examples

Open the bundled sample database and run a query (requires a Java runtime):

>>> import os
>>> import gcamreader
>>> data_dir = gcamreader.sample_data_dir()
>>> conn = gcamreader.LocalDBConn(
...     data_dir, "sample_basexdb"
... )
>>> query = gcamreader.parse_batch_query(
...     os.path.join(
...         data_dir,
...         "queries",
...         "query_land_reg32_basin235_gcam5p0.xml",
...     )
... )[0]
>>> df = conn.runQuery(query)
runQuery(query, scenarios=None, regions=None, warn_empty=True)[source]

Run a query on this connection.

Run the supplied query and return the result as a pandas DataFrame. This query will generally have been parsed from a GCAM queries XML file. The query can contain a list of regions parsed from the XML file. If present, query results will be filtered to this list of regions; otherwise, all regions will be included. The regions argument, if present, overrides the region filters parsed from the XML. Passing an empty list removes the region filter entirely.

Parameters:

  • query (Query) – A Query object.

  • scenarios (str | list[str] | None) – A list of scenarios to include in query results. If None, the last scenario in the database is used.

  • regions (str | list[str] | None) – A list of regions to filter query results to. See the description for the behavior when regions is None.

  • warn_empty (bool) – Whether to issue a warning to stderr if the result is empty.

Return type:

DataFrame | None

Returns:

A DataFrame of the query results, or None if the result was empty.

Examples

>>> import gcamreader
>>> conn = gcamreader.LocalDBConn(dbpath, dbfile)
>>> query = gcamreader.parse_batch_query(
...     "queries.xml"
... )[0]
>>> df = conn.runQuery(
...     query, scenarios=["Reference"], regions=["USA"]
... )
listScenariosInDB()[source]

List the scenarios contained in the GCAM database.

To run a query users typically need to know the names of the scenarios in the database. The result is a table with columns name, date, version, and fqName. The name and date are exactly as specified in the database. The fqName is the fully qualified scenario name which can be used in the scenarios argument of runQuery() to disambiguate scenario names. The version is the GCAM version tag used to generate the scenario.

Return type:

DataFrame | None

Returns:

A DataFrame of scenarios, or None if the query returned no results.

Examples

>>> import gcamreader
>>> conn = gcamreader.LocalDBConn(dbpath, dbfile)
>>> scenarios = conn.listScenariosInDB()
>>> list(scenarios["name"])
['Reference']
class gcamreader.querymi.RemoteDBConn(dbfile, username, password, address='localhost', port=8984, validatedb=True)[source]

Bases: object

Connection to a remote GCAM database.

A remote database connection communicates with a webserver using the BaseX REST API. The connection requires a server address, server port, username and password (configured in the server setup), and the database name on the remote server.

Parameters:
dbfile

The database file to query.

username

The username configured for the BaseX server.

password

The password configured for the BaseX server.

address

The server address (URL).

port

The port the server is running on.

Examples

Connect to a BaseX server and run a query (requires a running server):

>>> import gcamreader
>>> conn = gcamreader.RemoteDBConn(
...     dbfile="my_database",
...     username="user",
...     password="secret",
...     address="localhost",
...     port=8984,
... )
>>> query = gcamreader.parse_batch_query(
...     "queries.xml"
... )[0]
>>> df = conn.runQuery(query)
runQuery(query, scenarios=None, regions=None, warn_empty=True)[source]

Run a query on this connection.

Run the supplied query and return the result as a pandas DataFrame. This query will generally have been parsed from a GCAM queries XML file. The query can contain a list of regions parsed from the XML file. If present, query results will be filtered to this list of regions; otherwise, all regions will be included. The regions argument, if present, overrides the region filters parsed from the XML. Passing an empty list removes the region filter entirely.

Parameters:

  • query (Query) – A Query object.

  • scenarios (str | list[str] | None) – A list of scenarios to include in query results. If None, the last scenario in the database is used.

  • regions (str | list[str] | None) – A list of regions to filter query results to. See the description for the behavior when regions is None.

  • warn_empty (bool) – Whether to issue a warning to stderr if the result is empty.

Return type:

DataFrame | None

Returns:

A DataFrame of the query results, or None if the result was empty.

Examples

>>> import gcamreader
>>> conn = gcamreader.RemoteDBConn(
...     "my_database", "user", "secret"
... )
>>> query = gcamreader.parse_batch_query(
...     "queries.xml"
... )[0]
>>> df = conn.runQuery(query, regions=["USA"])
listScenariosInDB()[source]

List the scenarios contained in the GCAM database.

To run a query users typically need to know the names of the scenarios in the database. The result is a table with columns name, date, version, and fqName. The name and date are exactly as specified in the database. The fqName is the fully qualified scenario name which can be used in the scenarios argument of runQuery() to disambiguate scenario names. The version is the GCAM version tag used to generate the scenario.

Return type:

DataFrame | None

Returns:

A DataFrame of scenarios, or None if the query returned no results.

Examples

>>> import gcamreader
>>> conn = gcamreader.RemoteDBConn(
...     "my_database", "user", "secret"
... )
>>> scenarios = conn.listScenariosInDB()
>>> list(scenarios["name"])
['Reference']
gcamreader.querymi.importdata(dbspec, queries, scenarios=None, regions=None, warn_empty=False, suppress_gabble=True, miclasspath=None)[source]

Run a selection of queries against a database connection.

Run all of the queries in a GCAM queries file against a database connection and return a dictionary of query results indexed by query title.

Parameters:

  • dbspec (str | LocalDBConn | RemoteDBConn) – A database connection, or a string with the filename for a GCAM database.

  • queries (str | list[Query]) – Filename of a GCAM queries XML file, or the output of parse_batch_query() run on such a file.

  • scenarios (str | list[str] | None) – List of scenario names to include in the queries.

  • regions (str | list[str] | None) – List of regions to include in the queries.

  • warn_empty (bool) – Whether to print a warning if a query returns empty. The default is False.

  • suppress_gabble (bool) – Whether to suppress model interface console output when a connection is created from a filename.

  • miclasspath (str | None) – Java class path for the GCAM model interface when a connection is created from a filename.

Return type:

dict[str | None, DataFrame | None]

Returns:

A dictionary mapping each query title to its result DataFrame (or None for empty results).

Examples

Run every query in a batch file against a local database and access a result by its title (requires a Java runtime):

>>> import gcamreader
>>> results = gcamreader.importdata(
...     "/path/to/database_basexdb",
...     "queries.xml",
...     scenarios=["Reference"],
... )
>>> results["Crop Land Allocation"].head()

A previously created connection may be passed instead of a filename:

>>> conn = gcamreader.LocalDBConn(dbpath, dbfile)
>>> results = gcamreader.importdata(
...     conn, "queries.xml"
... )