|
|
To use the tests decide which tests are relevant to the analysis and will be used for masking data from the analysis. [For this example](direct-normal-broadband), all tests with an assessment of "Bad" are chosen, except for test 4 which is not in use. The tests used are [1, 2, 3, 5]. Test 6 is described as "Indeterminate" and will not be part of the mask applied to the data.
|
|
|
|
|
|
For this example, assume the data and companion QC fields are ten element arrays and equal to the values
|
|
|
|
|
|
import numpy as np
|
|
|
direct_normal_broadband = np.array([10, 9, -1, -0.1, 11, 20, 50, 110, 12200, 134])
|
|
|
qc_direct_normal_broadband = np.array([16, 16, 3, 1, 0, 0, 0, 0, 4, 32])
|
|
|
|
|
|
Next, create an array of tests to use in the mask to exclude values
|
|
|
|
|
|
tests = np.array([1, 2, 3, 5])
|
|
|
|
|
|
Create a single integer summarizing all the tests of interest from the 'tests' array. For the tests value defined mask_test is equal to 23.
|
|
|
|
|
|
mask_test = np.sum(np.left_shift(1, tests-1)) # mask_test = 23 for tests 1,2,3,5
|
|
|
print np.binary_repr(mask_test,width=16) # Print the binary representaion of the number
|
|
|
0000000000010111
|
|
|
|
|
|
Search the companion QC field for at least one of the tests to be tripped. The result of the search is an array of true-false values.
|
|
|
|
|
|
mask = qc_direct_normal_broadband & mask_test > 0
|
|
|
print mask
|
|
|
[ True True True True False False False False True False]
|
|
|
|
|
|
Then use the mask to replace values failing QC test with IEEE Not A Number to remove the data value from analysis
|
|
|
|
|
|
direct_normal_broadband[mask] = np.nan
|
|
|
print direct_normal_broadband
|
|
|
[ nan nan nan nan 11. 20. 50. 110. nan 134.]
|
|
|
|
|
|
If a different set of tests were desired the only part that needs to change is 'tests', the rest of the code is independent of datastream or analysis. This will also work with multi-dimensional data. |
|
|
\ No newline at end of file |