Analysis Request and Response Mappings
======================================

.. index:: Analysis API; Request and Response mapping

An analysis api takes in a
`NetworkAnalysisStreamRequest <api_reference.html#are-v1-analysis-NetworkAnalysisStreamRequest>`__
that can contain a list of
`NetworkAnalysisRequest <api_reference.html#are-v1-analysis-NetworkAnalysisRequest>`__.
Each ``NetworkAnalysisRequest`` can specify a list of descriptors in
each category (``Distribution Metrics Descriptor``,
``Ledger Export Descriptor``, ``SQL Query Descriptor``). Each
``NetworkAnalysisRequest`` takes in one network. A network can have
multiple terminals. For each ``NetworkAnalysisRequest``, there will be
one ``TerminalAnalysisResult`` per descriptor per terminal (unless
`sub-terminals <https://graphene.analyzere.net/analysis_api.html#subterminals>`__
are used).

To map each terminal analysis result to its network analysis stream
request:

`Find out which descriptor in the NetworkAnalysisRequest matches with
the descriptor of the
TerminalAnalysisResult <#mapping-terminalanalysisresults-to-requests-by-matching-descriptors>`__

If there are more then one ``NetworkAnalysisRequest`` in
``NetworkAnalysisStreamRequest``, then:

`Find out which NetworkAnalysisRequest the TerminalAnalysisResult is
associated
with. <#mapping-terminalanalysisresults-to-the-nth-networkanalysisrequest-using-network-index>`__

If there is more then one ``terminal`` in the ``network``, then:

`Find out which terminals the TerminalAnalysisResult is associated
with. <#mapping-terminalanalysisresults-to-their-respective-terminals-using-terminal-reference>`__

Note:

1. In the following examples, the ``network reference (id: 1, revision: 1)``
   references an example network with a single terminal.

2. If the response parsed message does not contain a particular singular element,
   the corresponding field in the parsed object is set to the default value for that field
   (ref `docs <https://developers.google.com/protocol-buffers/docs/proto3#default>`__).

3. In the following examples, we serialized responses with the following method to include all default values:

.. code:: python

   from google.protobuf.json_format import MessageToJson

   serialized_message_with_defaults = MessageToJson(
       response,
       including_default_value_fields=True
   )

Mapping TerminalAnalysisResults to requests by matching descriptors
-------------------------------------------------------------------

We have three types of analysis descriptors:
``Distribution Metrics Descriptor``, ``Ledger Export Descriptor``,
``SQL Query Descriptor``. A ``NetworkAnalysisRequest`` can specify
multiple descriptors of each type. Each ``TerminalAnalysisResult`` only
corresponds to one of the descriptors specified among the requests.
Matching the descriptor in the ``TerminalAnalysisResult`` with the
descriptor in the ``NetworkAnalysisRequest`` allow us map results to
requests.

Here are three examples showing a basic request and a corresponding
response for each type of analysis descriptors:

Distribution Metrics Descriptor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Network analysis request with single distribution metrics descriptor

**Request:**

.. code:: python

   distribution_metrics =  analysis_pb2.DistributionMetricsDescriptor(
       aggregation=analysis_pb2.AggregationDescriptor(aggregation_method="AEP"),
       windows=[analysis_pb2.ProbabilityWindow(min_probability=0.0, max_probability=1.0)]
   )


   metrics_analysis_request = analysis_api_pb2.NetworkAnalysisRequest(
       reference=data_management_api_pb2.NetworkReference(id=1, revision=1),
       context=analysis_api_pb2.Context(template_search_paths=["financial_model"]),
       distribution_metrics=[distribution_metrics]
   )

**Response:**

.. code:: json

   {
       "result": {
         "terminalReference": {
           "id": "1",
           "revision": "1"
         },
         "distributionMetrics": {
           "distributionMetricsDescriptor": {
             "aggregation": {
               "aggregationMethod": "AEP",
               "groupbyColumns": [],
               "currency": ""
             },
             "windows": [
               {
                 "maxProbability": 1.0,
                 "minProbability": 0.0
               }
             ],
             "thresholds": [],
             "fullCurve": false
           },
           "windowMetrics": [
             {
               "window": {
                 "maxProbability": 1.0,
                 "minProbability": 0.0
               },
               "mean": 5388173.281133945,
               "max": 7346375.170700987,
               "min": 3294584.7997547477,
               "standardDeviation": 1296044.6495295493
             }
           ],
           "grouping": [],
           "probabilities": []
         },
         "networkIndex": "0",
         "subterminalIndex": "0",
         "trial_range": {
           "trial_begin": 1,
           "trial_count": 100
         }
       }
   }

Ledger Exports Descriptor
~~~~~~~~~~~~~~~~~~~~~~~~~

network analysis request with single ledger exports descriptor

**Request:**

.. code:: python

   ledger_export = analysis_pb2.LedgerExportDescriptor(
       columns=["ModelCode", "PerilCode"]
   )

   export_analysis_request = analysis_api_pb2.NetworkAnalysisRequest(
       reference=data_management_api_pb2.NetworkReference(id=1, revision=1),
       context=analysis_api_pb2.Context(template_search_paths=["financial_model"]),
       ledger_exports=[ledger_export]
   )

**Response:**

.. code:: json

   {
       "result": {
         "terminalReference": {
           "id": "1",
           "revision": "1"
         },
         "ledgerExport": {
           "ledgerExportDescriptor": {
             "columns": [
               "ModelCode",
               "PerilCode"
             ],
             "occurrenceKeyColumn": "",
             "includeNodePaths": false,
             "currency": ""
           },
           "fileObjectReference": {
             "uri": "s3://<bucket_name>/exports/ledgers/<path_to_prefix>/",
             "isCollection": true,
             "format": "PARQUET"
           }
         },
         "networkIndex": "0",
         "subterminalIndex": "0",
         "trial_range": {
           "trial_begin": 1,
           "trial_count": 100
         }
       }
   }

SQL Query Descriptor
~~~~~~~~~~~~~~~~~~~~

network analysis request with single sql queries descriptor

**Requests:**

.. code:: python

   ledger_sql_query = analysis_pb2.LedgerSQLQueryDescriptor(
       columns=["Value"],
       queries=[
           analysis_pb2.SQLQuery(
               query="select Trial, SUM(value) from LEDGER group by Trial",
               format=analysis_pb2.SQLQuery.CSV,
               inline=False,
           )
       ],
   )

   export_analysis_request = analysis_api_pb2.NetworkAnalysisRequest(
       reference=data_management_api_pb2.NetworkReference(id=1, revision=1),
       context=analysis_api_pb2.Context(template_search_paths=["financial_model"]),
       ledger_sql_queries=[ledger_sql_query]
   )

**Response:**

.. code:: json

   {
       "result": {
         "terminalReference": {
           "id": "1",
           "revision": "1"
         },
         "ledgerSqlQuery": {
           "ledgerSqlQueryDescriptor": {
             "columns": [
               "Value"
             ],
             "queries": [
               {
                 "query": "select Trial, SUM(value) from LEDGER group by Trial",
                 "format": "CSV",
                 "inline": false
               }
             ],
             "currency": ""
           },
           "queryResult": {
             "fileObjectReference": {
               "format": "CSV",
               "uri": "s3://<bucket>/exports/sql_queries/<path_to_file>",
               "isCollection": false
             }
           },
           "queryIndex": "0"
         },
         "networkIndex": "0",
         "subterminalIndex": "0",
         "trial_range": {
           "trial_begin": 1,
           "trial_count": 100
         }
       }
   }

Mapping TerminalAnalysisResults to the nth NetworkAnalysisRequest using network index
-------------------------------------------------------------------------------------

When requesting a number of ``NetworkAnalysisRequests`` in a
``NetworkAnalysisStreamRequest``, the result field ``network_index`` in
each ``TerminalAnalysisResult`` indicates which
``NetworkAnalysisRequest`` it is associated with. E.g.:

We have made two ``NetworkAnalysisRequests``:
``metrics_analysis_request1``, ``metrics_analysis_request2``. And
requested them in the order that the ``network_index`` of
``metrics_analysis_request1`` is ``0``, and the ``network_index`` of
``metrics_analysis_request2`` is ``1``.

**Request:**

.. code:: python

   metrics_analysis_request1 = analysis_api_pb2.NetworkAnalysisRequest(
       reference=data_management_api_pb2.NetworkReference(id=1, revision=1),
       context=analysis_api_pb2.Context(template_search_paths=["financial_model"]),
       distribution_metrics=[distribution_metrics_1]
   )
   metrics_analysis_request2 = analysis_api_pb2.NetworkAnalysisRequest(
       reference=data_management_api_pb2.NetworkReference(id=1, revision=1),
       context=analysis_api_pb2.Context(template_search_paths=["financial_model"]),
       distribution_metrics=[distribution_metrics_2]
   )

   analysis_stream_request = analysis_api_pb2.NetworkAnalysisStreamRequest(
       trial_range=trial_range,
       network_analysis_requests=[metrics_analysis_request1, metrics_analysis_request2],
   )

**Response 1:**

.. code:: json

   {
       "result": {
       "networkIndex": "0",
       "terminalReference": {
         "id": "1",
         "revision": "1"
       },
       "...": "...."
       }
   }

**Response 2:**

.. code:: json

   {
       "result": {
         "network_index": "1",
         "terminalReference": {
           "id": "2",
           "revision": "2"
         },
       "...": "..."
       }
   }

The order of above responses is not guaranteed. The ``Response 1`` has
``network_index: 0`` which corresponds to the
``metrics_analysis_request1``. The ``Response 2`` has
``network_index: 1`` which corresponds to the
``metrics_analysis_request2`` in the ``analysis_stream_request``.

Mapping TerminalAnalysisResults to their respective terminals using terminal reference
--------------------------------------------------------------------------------------

For each ``NetworkAnalysisRequest``, a network is specified to do the
analysis on. A network can contain multiple terminal nodes where each
terminal produces one or more TerminalAnalysisResults. The result field
``terminal_reference`` indicates which terminal the
``TerminalAnalysisResult`` is associated with.

Using the same example as
`above <#mapping-terminalanalysisresults-to-the-nth-networkanalysisrequest-using-network-index>`__,
both ``Response 1`` and ``Response 2`` have the same
``terminal_reference`` because the network ``(id: 1, revision: 1)`` that
is used in both ``NetworkAnalysisRequests`` only has a single terminal.
If we change the example to use a network that has ``N`` terminal nodes
while keep the rest of the request the same, then we would expect
``2 * N`` number of ``TerminalAnalysisResults`` returned.