Data quality issues are often found some time after data is collected, ingested, archived and then delivered to an end user. The Data Quality Report (DQR) web service allows a user application to automatically query the DQR database in order to retrieve the most up-to-date information regarding the quality of their data holdings. The service returns a list of time blocks (start/end time pairs) that delineate when a user should not use or should be suspicious of their ARM data. Optionally, additional information about the DQR responsible for the time blocks in the report can be returned. This might include the DQR identifier string and the subject details.
## Build the Query
Generate a URL with the datastream and variable names encoded as parameters to the webservice.
For example, to search for DQRs affecting the datastream **sgplssondeC1.c1** and the variable **rh**, use this URL:
When submitted, you will see output something like this:
<pre>
944199060|944285400
1031333220|1031338440
1031290020|1031296680
1033644000|1033651200
1126265520|1126272720
</pre>
The default is a list of start and end times in seconds since Jan-1-1970, the same time units used in ARM data files. You can specify exactly the type of DQR information you would like returned, the time format, and whether you are interested in known incorrect (aka red), suspicious (aka yellow), or both types of DQR.
For example, if you would like to see the DQR id and prefer to see time as encoded in ARM filenames instead, add the dqrfields and timeformat parameters to the URL as so:
In the next sections we will delve into retrieving machine readable data quality information using web queries from several popular programming languages. Full documentation of the web service options can be found in [[Parameter Descriptions]]. These examples convey the basic techniques in performing a query and parsing the simple delimited response. For fully unattended code, it is advisable to check error codes from all connection attempts to ensure that:
* web service host is reachable (no network issues)
* valid parameters were passed
* web service and associated database is up and running
Examples provided in this repository will help you check that these conditions are met. By doing these checks, you can be confident that a blank response is due to an absence of DQRs rather than a problem with the service or the way it is being called.
