Protocol Documentation

Table of Contents

Top

proto/financial_model/common_internal.proto

Error

internal errors TODO: For ylt generator, promote trial range to parent message and/or add job number so that errors can be reported with more info

Field Type Label Description
code Error.Code
msg string

STSCredentials

Field Type Label Description
aws_access_key_id string
aws_secret_session_key string
aws_session_token string

SqsMessageRef

Field Type Label Description
bucket string
key string

Error.Code

Name Number Description
UNSPECIFIED 0 The default if an unknown enum value is provided.
ERROR_INTERNAL 1 An error that may get retried if lambda retry limits have not been exhausted; for informational purposes only
ERROR_UNHANDLED 2 A system error occurred that was unexpected; for informational purposes only; the system may retry TODO: currently ERROR_UNHANDLED is not used anywhere, verify if GR-1936 need it if not, remove this error code.
ERROR_VALIDATION 3 A problem with input that prevents request completion and which should be returned via the API
ERROR_PERMANENT 4 A problem with the system that prevents request completion; should be communicated to the user immediately; not user facing; used for errors that need to hide detail from the user "permanent" implies not retryable although this may be used for both cases; could have another code to distinguish these if needed
ERROR_SYSTEM 5 A problem with the system that prevents request completion; should be communicated to the user immediately; user should be provided with error details; may or may not be retryable; could have an additional code here to distinguish those cases

Top

proto/financial_model/financial_model_api.proto

AsyncAnalysisServiceRequest

Internal request passed to the async analysis service via SQS

Field Type Label Description
short_request_id string
global_request_id string
tenant_info TenantInfo
access_token string

MetricsStreamHeader

Field Type Label Description
trial_range are.v1.common.TrialRange
graph_key string

MetricsStreamReply

The root level output

Field Type Label Description
header MetricsStreamHeader
error are.v1.common.Error Deprecated.
vertex_metrics_reply VertexMetricsStreamReply
status are.v1.common.Status

MetricsStreamRequest

The root level input

Field Type Label Description
trial_range are.v1.common.TrialRange
graph are.v1.financial_model_graph.Graph
tenant_info TenantInfo
vertex_metrics_requests VertexMetricsRequest repeated
rate_table_uri string
oep_cometrics_mapping string Deprecated.
enable_aggregate_merging bool
enable_packed_row_groups bool
enable_export_staging bool

MetricsStreamRequestKey

Field Type Label Description
key string

TenantInfo

Field Type Label Description
tenant_name string
bucket_name string
client_role_arn string
vss_tenant_name string

VertexMetricsRequest

Field Type Label Description
vertex_index uint64 The index of the vertex in the financial model graph for which to aggregate the ledger.
distribution_metrics are.v1.analysis.DistributionMetricsDescriptor repeated
ledger_exports are.v1.analysis.LedgerExportDescriptor repeated
ledger_sql_queries are.v1.analysis.LedgerSQLQueryDescriptor repeated
distribution_cometrics are.v1.analysis.CoMetricsVerticesDescriptor repeated

VertexMetricsStreamReply

Field Type Label Description
vertex_index uint64
metrics are.v1.analysis.DistributionMetricsResult
export_path are.v1.analysis.LedgerExportResult
ledger_sql_query_result are.v1.analysis.LedgerSQLQueryResult
cometrics are.v1.analysis.DistributionCoMetricsVertexResult

Metrics

Method Name Request Type Response Type Description
GetMetrics MetricsStreamRequestKey MetricsStreamReply stream

Top

proto/financial_model/financial_model_graph.proto

AggregateAttachment

Field Type Label Description
attachment double
currency string Optional currency in which the attachment is defined. The attachment will be translated to the base currency using the rate table provided on request if this is specified.

AggregateLimit

Field Type Label Description
limit double
currency string Optional currency in which the limit is defined. The limit will be translated to the base currency using the rate table provided on request if this is specified.

Edge

Field Type Label Description
from_vertex uint64
to_vertex uint64
path_labelling_weight uint64

FinancialModelGraphFieldOptions

Field Type Label Description
is_required bool Note that is_required defaults to False if not set

FinancialModelGraphMessageOptions

Field Type Label Description
may_start_label_path bool optional The vertices need to be able to generate ledger rows that are considered new (originating from the vertex, not a modified version of a row that originates elsewhere) and not control records.

Graph

Field Type Label Description
id string
edges Edge repeated
vertices Vertex repeated

InnerJoin

Field Type Label Description
join_record_type string
join_on string repeated
op InnerJoin.Operation

LargestOccFilter

Field Type Label Description
count uint64
invert_order bool
invert_selection bool

LoadELTDataAndSimulate

Field Type Label Description
paths string repeated
model LoadELTDataAndSimulate.Model
currency string

LoadELTDataAndSimulate.Model

Field Type Label Description
seasonality LoadELTDataAndSimulate.Model.Seasonality
frequency LoadELTDataAndSimulate.Model.Frequency
seed bool

LoadELTDataAndSimulate.Model.Frequency

Field Type Label Description
distribution_type LoadELTDataAndSimulate.Model.Frequency.Type
mean double mean is not used until parametric ELT Loss Sets are implemented
index_of_dispersion double

LoadELTDataAndSimulate.Model.Seasonality

Field Type Label Description
type LoadELTDataAndSimulate.Model.Seasonality.Type
subtype LoadELTDataAndSimulate.Model.Seasonality.EmpiricalSubtype
pairs LoadELTDataAndSimulate.Model.Seasonality.EmpiricalPair repeated
min_time double
max_time double
interpolate bool

LoadELTDataAndSimulate.Model.Seasonality.EmpiricalPair

Field Type Label Description
time double
probability double

LoadFactor

Field Type Label Description
factor_type_name string
match_by string repeated

LoadFromFile

Field Type Label Description
path string
occurrence_key_column string
currency string Optional currency in which the values contained in the file are defined. The values will be translated to the base currency using the rate table provided on request if this is specified.
currency_column string Optional ledger column that specifies currency of the each individual record The values of each record will be translated to the base currency Mutually exclusive with the currency field

LoadFromFile.Term

Field Type Label Description
column string
value string

LookupFactor

Field Type Label Description
table_path string
match_on string repeated
factor string
default_value double

MetadataFilter

Field Type Label Description
expression string

MultiplicativeInteraction

Field Type Label Description
basis_record_type string

OccurrenceAttachment

Field Type Label Description
attachment double
currency string Optional currency in which the attachment is defined. The attachment will be translated to the base currency using the rate table provided on request if this is specified.

OccurrenceFranchiseDeductible

Field Type Label Description
franchise_deductible double
currency string Optional currency in which the franchise deductible is defined. The franchise deductible will be translated to the base currency using the rate table provided on request if this is specified.

OccurrenceLimit

Field Type Label Description
limit double
currency string Optional currency in which the limit is defined. The limit will be translated to the base currency using the rate table provided on request if this is specified.

OffsetTime

Field Type Label Description
value float

PreferentialAttachment

Field Type Label Description
attachment double
erode_by_columns string repeated Optional list of columns to erode over. This will always include the "Trial" column.
span_columns string repeated Optional list of columns to span over. This will always include the "Trial" column.
preferred_column string Optional column. This will always include the "Trial" column.
preferred_values string repeated Column which contains values that will be used for preferential erosion
currency string Optional currency in which the attachment is defined. The attachment will be translated to the base currency using the rate table provided on request if this is specified.
invert bool invert preferential order, e.g. attachment applied to preferred column last

Projection

Field Type Label Description
columns string repeated

Record

Field Type Label Description
time double
type string
value double
currency string Optional currency in which the value is defined. The value will be translated to the base currency using the rate table provided on request if this is specified.

RecordTypeFilter

Field Type Label Description
op RecordTypeFilter.Operation
value string

RecordTypePrefixFilter

Field Type Label Description
op RecordTypePrefixFilter.Operation
value string

Scale

Field Type Label Description
factor double

ScopedAttachment

Field Type Label Description
attachment double
erode_by_columns string repeated Optional list of columns to erode over. This will always include the "Trial" column.
span_columns string repeated Optional list of columns to span over. This will always include the "Trial" column.
currency string Optional currency in which the limit is defined. The limit will be translated to the base currency using the rate table provided on request if this is specified.

ScopedLimit

Field Type Label Description
limit double
erode_by_columns string repeated Optional list of columns to erode over. This will always include the "Trial" column.
span_columns string repeated Optional list of columns to span over. This will always include the "Trial" column.
currency string Optional currency in which the limit is defined. The limit will be translated to the base currency using the rate table provided on request if this is specified.

SetRecordType

Field Type Label Description
value string

SyntheticLedgerGenerator

Emits a random ledger with the specified number of records. For internal use only! (stress testing / benchmarking)

Field Type Label Description
records_per_trial uint64 Number of records to emit per trial

TimeFilter

Field Type Label Description
op TimeFilter.Operation
value double

TimePeriodProportion

Field Type Label Description
period_start double
period_end double
time_elapsed_basis bool

Trigger

Field Type Label Description
threshold double The trigger threshold.
op Trigger.Operation
control_record_type string Required record type string that is used to identify control loss records.
accumulation_class_columns string repeated Optional list of columns that define the accumulation class. This will always include the "Trial" column.
accumulation_prefix_class_columns string repeated Optional list of columns that define the accumulation prefix class. This will always include the "Trial" column.
exclusive bool Optional field to exclude the current prefix class from the threshold calculation.
currency string Optional currency in which the threshold is defined. The threshold will be translated to the base currency using the rate table provided on request if this is specified.

Vertex

Field Type Label Description
terms google.protobuf.Any
label string

InnerJoin.Operation

Name Number Description
UNSPECIFIED_NO_OPERATION 0 > For enums, the default value is the first defined enum value, which must be 0. https://protobuf.dev/programming-guides/proto3/#enum
MULTIPLY 1

LoadELTDataAndSimulate.Model.Frequency.Type

Name Number Description
UNSPECIFIED_TYPE 0
POISSON 1
BINOMIAL 2
NEGATIVE_BINOMIAL 3

LoadELTDataAndSimulate.Model.Seasonality.EmpiricalSubtype

Name Number Description
UNSPECIFIED_SUBTYPE 0
NON_CUMULATIVE 1
CUMULATIVE 2

LoadELTDataAndSimulate.Model.Seasonality.Type

Name Number Description
UNSPECIFIED_TYPE 0
UNIFORM 1
EMPIRICAL 2

RecordTypeFilter.Operation

Name Number Description
UNSPECIFIED 0
EQUAL 1
NOT_EQUAL 2

RecordTypePrefixFilter.Operation

Name Number Description
UNSPECIFIED 0
EQUAL 1
NOT_EQUAL 2

TimeFilter.Operation

Name Number Description
UNSPECIFIED 0
LESS_THAN 1
LESS_THAN_EQUAL 2
GREATER_THAN 3
GREATER_THAN_EQUAL 4

Trigger.Operation

Name Number Description
UNSPECIFIED 0
LESS_THAN 1
LESS_THAN_EQUAL 2
GREATER_THAN 3
GREATER_THAN_EQUAL 4

File-level Extensions

| Extension | Type | Base | Number | Description | | ——— | —- | —- | —— | ———– | | fm_options | FinancialModelGraphFieldOptions | .google.protobuf.FieldOptions | 1075 | | | fm_message_options | FinancialModelGraphMessageOptions | .google.protobuf.MessageOptions | 1075 | |

Top

proto/financial_model/metrics_engine.proto

GroupInfo

Field Type Label Description
number uint32 The number of this group (One based)
total uint32 Total groups in this result

MetricsEngineReply

Metrics engine output

Field Type Label Description
result MetricsEngineResult
error are.v1.common_internal.Error

MetricsEngineRequest

Metrics engine input

Field Type Label Description
request_id string
trial_range are.v1.common.TrialRange The range of the entire analysis
trials_per_chunk uint64 The YLT generator trial chunk size
graph_key string
output_channel_name string
vertex_index uint64
tenant_info are.v1.financial_model.TenantInfo
is_terminated_key string If key exists in redis, request has been terminated
metrics_descriptor are.v1.analysis.DistributionMetricsDescriptor
cometrics_descriptor are.v1.analysis.CoMetricsVertexDescriptor
group_info GroupInfo YLT group info
grouping are.v1.common.Cell repeated
oep_cometrics_mapping string Deprecated.

MetricsEngineResult

Field Type Label Description
request_id string
vertex_index uint64
group_info GroupInfo
metrics_result are.v1.analysis.DistributionMetricsResult
cometrics_result are.v1.analysis.DistributionCoMetricsVertexResult

Top

proto/financial_model/s3_transfer.proto

S3TransferJob

Field Type Label Description
request_id string
tenant_bucket string
internal_bucket string
prefix string
failures uint64 optional
part uint64 optional
part_count uint64 optional

Top

proto/financial_model/sql_service.proto

SQLServiceReply

SQLService output

Field Type Label Description
request_id string
result SQLServiceResult
error are.v1.common_internal.Error

SQLServiceRequest

SQLService input

Field Type Label Description
request_id string
input_parquet_uri string
vertex_index uint64
output_channel_name string
tenant_info are.v1.financial_model.TenantInfo
ledger_sql_query_descriptor are.v1.analysis.LedgerSQLQueryDescriptor
is_terminated_key string If key exists in redis, request has been terminated
sts_credentials are.v1.common_internal.STSCredentials

SQLServiceResult

Field Type Label Description
vertex_index uint64
ledger_sql_results are.v1.analysis.LedgerSQLQueryResult repeated

Top

proto/financial_model/ylt_generator.proto

CurrencyExchangeRate

Field Type Label Description
currency string The name or identifier of the currency. ie. "USD"
rate double The rate of the currency relative to the base (empty) currency.

CurrencyExchangeRateTable

Field Type Label Description
rates CurrencyExchangeRate repeated A list of exchange rates for currency exchange

SimulationDescriptor

Field Type Label Description
graph are.v1.financial_model_graph.Graph
vertex_descriptors SimulationDescriptor.VertexDescriptor repeated The list of vertex descriptors for the different vertices.
cancelled bool Deprecated. Indicates the simulation request has been cancelled DEPRECATED: cancellation can only be tested by checking existence of the key specified by YLTGeneratorRequest::is_terminated_key in Redis
currency_exchange_rate_table CurrencyExchangeRateTable A list of exchange rates for currency exchange

SimulationDescriptor.VertexDescriptor

Descriptors for a specific vertex.

Field Type Label Description
vertex_index uint64 The vertex index.
ylt_reqs are.v1.analysis.AggregationDescriptor repeated The aggregation descriptors for the vertex, i.e. the different YLTs to be generated for this vertex.
ledger_exports are.v1.analysis.LedgerExportDescriptor repeated The ledger export descriptors, i.e. the different exports to be done for this vertex.

YLTGeneratorReply

YLT generator output

Field Type Label Description
result YLTGeneratorResult
error are.v1.common_internal.Error
warning_message string optional

YLTGeneratorRequest

YLT generator input

Field Type Label Description
part_info YLTGeneratorRequest.PartInfo The part information
trial_range are.v1.common.TrialRange
simulation_descriptor_key string
tenant_info are.v1.financial_model.TenantInfo
complete_channel_name string
remove_zeros bool
is_terminated_key string If key exists in redis, request has been terminated
sts_credentials are.v1.common_internal.STSCredentials
enable_aggregate_merging bool
enable_packed_row_groups bool
activity_id string
enable_export_staging bool

YLTGeneratorRequest.PartInfo

Field Type Label Description
number uint32 The number of this part (One based)
total uint32 Total parts involved in this task

YLTGeneratorRequestBatch

Field Type Label Description
tenant_info are.v1.financial_model.TenantInfo
simulation_descriptor_key string
trial_ranges are.v1.common.TrialRange repeated
complete_channel_name string
is_terminated_key string If key exists in redis, request has been terminated
sts_credentials are.v1.common_internal.STSCredentials
enable_aggregate_merging bool
enable_packed_row_groups bool
enable_export_staging bool

YLTGeneratorResult

Field Type Label Description
trial_range are.v1.common.TrialRange
vertex_results YLTGeneratorResult.VertexResult repeated List of results for multiple vertices.

YLTGeneratorResult.VertexResult

The direct YLT generator results for a single vertex.

Field Type Label Description
vertex_index uint64 The vertex index.
export_paths are.v1.analysis.LedgerExportResult repeated The list of legder export results for this vertex.

YLTResultField

Field Type Label Description
grouping are.v1.common.Cell repeated

YLTResultKey

YLT generator redis messages

Field Type Label Description
graph_key string The S3 key of the graph for which the results are.
vertex_index uint64 The vertex index for the YLT.
ylt_group are.v1.analysis.AggregationDescriptor The aggregation descriptor for the YLT.
trial_range are.v1.common.TrialRange The trial range for the YLT.

YLTResultValue

Field Type Label Description
values double repeated

Scalar Value Types

.proto Type Notes C++ Java Python Go C# PHP Ruby
double double double float float64 double float Float
float float float float float32 float float Float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int int32 int integer Bignum or Fixnum (as required)
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long int64 long integer/string Bignum
uint32 Uses variable-length encoding. uint32 int int/long uint32 uint integer Bignum or Fixnum (as required)
uint64 Uses variable-length encoding. uint64 long int/long uint64 ulong integer/string Bignum or Fixnum (as required)
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int int32 int integer Bignum or Fixnum (as required)
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long int64 long integer/string Bignum
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int uint32 uint integer Bignum or Fixnum (as required)
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long uint64 ulong integer/string Bignum
sfixed32 Always four bytes. int32 int int int32 int integer Bignum or Fixnum (as required)
sfixed64 Always eight bytes. int64 long int/long int64 long integer/string Bignum
bool bool boolean boolean bool bool boolean TrueClass/FalseClass
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode string string string String (UTF-8)
bytes May contain any arbitrary sequence of bytes. string ByteString str []byte ByteString string String (ASCII-8BIT)