Analysis Request and Response Mappings¶
An analysis api takes in a
NetworkAnalysisStreamRequest
that can contain a list of
NetworkAnalysisRequest.
Each NetworkAnalysisRequest can specify a list of descriptors in
each category (Distribution Metrics Descriptor,
Ledger Export Descriptor, SQL Query Descriptor). Each
NetworkAnalysisRequest takes in one network. A network can have
multiple terminals. For each NetworkAnalysisRequest, there will be
one TerminalAnalysisResult per descriptor per terminal (unless
sub-terminals
are used).
To map each terminal analysis result to its network analysis stream request:
If there are more then one NetworkAnalysisRequest in
NetworkAnalysisStreamRequest, then:
Find out which NetworkAnalysisRequest the TerminalAnalysisResult is associated with.
If there is more then one terminal in the network, then:
Find out which terminals the TerminalAnalysisResult is associated with.
Note:
In the following examples, the
network reference (id: 1, revision: 1)references an example network with a single terminal.If the response parsed message does not contain a particular singular element, the corresponding field in the parsed object is set to the default value for that field (ref docs).
In the following examples, we serialized responses with the following method to include all default values:
from google.protobuf.json_format import MessageToJson
serialized_message_with_defaults = MessageToJson(
response,
including_default_value_fields=True
)
Mapping TerminalAnalysisResults to requests by matching descriptors¶
We have three types of analysis descriptors:
Distribution Metrics Descriptor, Ledger Export Descriptor,
SQL Query Descriptor. A NetworkAnalysisRequest can specify
multiple descriptors of each type. Each TerminalAnalysisResult only
corresponds to one of the descriptors specified among the requests.
Matching the descriptor in the TerminalAnalysisResult with the
descriptor in the NetworkAnalysisRequest allow us map results to
requests.
Here are three examples showing a basic request and a corresponding response for each type of analysis descriptors:
Distribution Metrics Descriptor¶
Network analysis request with single distribution metrics descriptor
Request:
distribution_metrics = analysis_pb2.DistributionMetricsDescriptor(
aggregation=analysis_pb2.AggregationDescriptor(aggregation_method="AEP"),
windows=[analysis_pb2.ProbabilityWindow(min_probability=0.0, max_probability=1.0)]
)
metrics_analysis_request = analysis_api_pb2.NetworkAnalysisRequest(
reference=data_management_api_pb2.NetworkReference(id=1, revision=1),
context=analysis_api_pb2.Context(template_search_paths=["financial_model"]),
distribution_metrics=[distribution_metrics]
)
Response:
{
"result": {
"terminalReference": {
"id": "1",
"revision": "1"
},
"distributionMetrics": {
"distributionMetricsDescriptor": {
"aggregation": {
"aggregationMethod": "AEP",
"groupbyColumns": [],
"currency": ""
},
"windows": [
{
"maxProbability": 1.0,
"minProbability": 0.0
}
],
"thresholds": [],
"fullCurve": false
},
"windowMetrics": [
{
"window": {
"maxProbability": 1.0,
"minProbability": 0.0
},
"mean": 5388173.281133945,
"max": 7346375.170700987,
"min": 3294584.7997547477,
"standardDeviation": 1296044.6495295493
}
],
"grouping": [],
"probabilities": []
},
"networkIndex": "0",
"subterminalIndex": "0",
"trial_range": {
"trial_begin": 1,
"trial_count": 100
}
}
}
Ledger Exports Descriptor¶
network analysis request with single ledger exports descriptor
Request:
ledger_export = analysis_pb2.LedgerExportDescriptor(
columns=["ModelCode", "PerilCode"]
)
export_analysis_request = analysis_api_pb2.NetworkAnalysisRequest(
reference=data_management_api_pb2.NetworkReference(id=1, revision=1),
context=analysis_api_pb2.Context(template_search_paths=["financial_model"]),
ledger_exports=[ledger_export]
)
Response:
{
"result": {
"terminalReference": {
"id": "1",
"revision": "1"
},
"ledgerExport": {
"ledgerExportDescriptor": {
"columns": [
"ModelCode",
"PerilCode"
],
"occurrenceKeyColumn": "",
"includeNodePaths": false,
"currency": ""
},
"fileObjectReference": {
"uri": "s3://<bucket_name>/exports/ledgers/<path_to_prefix>/",
"isCollection": true,
"format": "PARQUET"
}
},
"networkIndex": "0",
"subterminalIndex": "0",
"trial_range": {
"trial_begin": 1,
"trial_count": 100
}
}
}
SQL Query Descriptor¶
network analysis request with single sql queries descriptor
Requests:
ledger_sql_query = analysis_pb2.LedgerSQLQueryDescriptor(
columns=["Value"],
queries=[
analysis_pb2.SQLQuery(
query="select Trial, SUM(value) from LEDGER group by Trial",
format=analysis_pb2.SQLQuery.CSV,
inline=False,
)
],
)
export_analysis_request = analysis_api_pb2.NetworkAnalysisRequest(
reference=data_management_api_pb2.NetworkReference(id=1, revision=1),
context=analysis_api_pb2.Context(template_search_paths=["financial_model"]),
ledger_sql_queries=[ledger_sql_query]
)
Response:
{
"result": {
"terminalReference": {
"id": "1",
"revision": "1"
},
"ledgerSqlQuery": {
"ledgerSqlQueryDescriptor": {
"columns": [
"Value"
],
"queries": [
{
"query": "select Trial, SUM(value) from LEDGER group by Trial",
"format": "CSV",
"inline": false
}
],
"currency": ""
},
"queryResult": {
"fileObjectReference": {
"format": "CSV",
"uri": "s3://<bucket>/exports/sql_queries/<path_to_file>",
"isCollection": false
}
},
"queryIndex": "0"
},
"networkIndex": "0",
"subterminalIndex": "0",
"trial_range": {
"trial_begin": 1,
"trial_count": 100
}
}
}
Mapping TerminalAnalysisResults to the nth NetworkAnalysisRequest using network index¶
When requesting a number of NetworkAnalysisRequests in a
NetworkAnalysisStreamRequest, the result field network_index in
each TerminalAnalysisResult indicates which
NetworkAnalysisRequest it is associated with. E.g.:
We have made two NetworkAnalysisRequests:
metrics_analysis_request1, metrics_analysis_request2. And
requested them in the order that the network_index of
metrics_analysis_request1 is 0, and the network_index of
metrics_analysis_request2 is 1.
Request:
metrics_analysis_request1 = analysis_api_pb2.NetworkAnalysisRequest(
reference=data_management_api_pb2.NetworkReference(id=1, revision=1),
context=analysis_api_pb2.Context(template_search_paths=["financial_model"]),
distribution_metrics=[distribution_metrics_1]
)
metrics_analysis_request2 = analysis_api_pb2.NetworkAnalysisRequest(
reference=data_management_api_pb2.NetworkReference(id=1, revision=1),
context=analysis_api_pb2.Context(template_search_paths=["financial_model"]),
distribution_metrics=[distribution_metrics_2]
)
analysis_stream_request = analysis_api_pb2.NetworkAnalysisStreamRequest(
trial_range=trial_range,
network_analysis_requests=[metrics_analysis_request1, metrics_analysis_request2],
)
Response 1:
{
"result": {
"networkIndex": "0",
"terminalReference": {
"id": "1",
"revision": "1"
},
"...": "...."
}
}
Response 2:
{
"result": {
"network_index": "1",
"terminalReference": {
"id": "2",
"revision": "2"
},
"...": "..."
}
}
The order of above responses is not guaranteed. The Response 1 has
network_index: 0 which corresponds to the
metrics_analysis_request1. The Response 2 has
network_index: 1 which corresponds to the
metrics_analysis_request2 in the analysis_stream_request.
Mapping TerminalAnalysisResults to their respective terminals using terminal reference¶
For each NetworkAnalysisRequest, a network is specified to do the
analysis on. A network can contain multiple terminal nodes where each
terminal produces one or more TerminalAnalysisResults. The result field
terminal_reference indicates which terminal the
TerminalAnalysisResult is associated with.
Using the same example as
above,
both Response 1 and Response 2 have the same
terminal_reference because the network (id: 1, revision: 1) that
is used in both NetworkAnalysisRequests only has a single terminal.
If we change the example to use a network that has N terminal nodes
while keep the rest of the request the same, then we would expect
2 * N number of TerminalAnalysisResults returned.