.. index:: Core Operations; Primitives; Scaled Load .. _primitive-scaled-load: Scaled Load ================================== The Scaled load template allows you to apply a factor value (such as order of magnitude, or negative value) and currency type to a set of ledgers. If occurency_key_column is included, it will be used to match occurrences for merging values together. Suppose these ledgers have been created in S3: .. code-block:: text s3://example_bucket/uploads/ledgers/test-files/parquet/general/simple.parquet s3://example_bucket/uploads/ledgers/test-files/parquet/general/one.parquet The general structure of the Scaled Load template in Graphene is: .. code-block:: json { "_schema": "ScaledLoad_1.0", "paths": ["s3://example_bucket/uploads/ledgers/test-files/parquet/general/simple.parquet","s3://example_bucket/uploads/ledgers/test-files/parquet/general/one.parquet"], "factor": -1, "currency": "DOUBLE", } or .. code-block:: json { "_schema": "ScaledLoad_1.0", "paths": ["s3://example_bucket/uploads/ledgers/test-files/parquet/general/record_level_currency.parquet"], "factor": -1, "currency_colunn": "Currency", } Parameters ---------- The parameters are defined as follows: +---------------------------+----------+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter Name | Required | Type | Description | +===========================+==========+=================+=======================================================================================================================================================================+ | ``factor`` | Yes | ``double`` | Amount that the result will by multiplied by. For example, if you have a factor of -1 then all numbers in the result will be negative. | +---------------------------+----------+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``currency`` | No | ``string`` | Currency that the ledger(s) are defined in, will be used to convert into the base currency. | +---------------------------+----------+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``paths`` | Yes | ``string list`` | Ledger(s) path location on s3 | +---------------------------+----------+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``occurency_key_column`` | No | ``string`` | Name of column to use as the occurrence key. If not provided, no occurrence key is used. | +---------------------------+----------+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``currency_column`` | No | ``string`` | Name of column used to specify currencies of the individual records. If proficed, will be used to convert records into base currency. | +---------------------------+----------+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Behavior --------- In the first case, all ledgers located at the ``path`` will be multiplied by ``factor`` and be converted from their ``currency`` value into the base currency. In the second case, each record in the ledger will be multiplied by ``factor`` and be converted from the currency stored in ``currency_column`` into the base currency. If the ``currency`` and ``currency_column`` values was omitted, it will still apply the ``factor`` value to the ledger(s). .. note:: it is not allowed to specify both ``currency`` and ``currency_column`` at the same time. Multi-partition files are supported. Example --------- User uploads ledger, ``currency`` is set to "DOUBLE" and ``factor`` is set to -1. Ledger 1 (simple.parquet): +---------+--------+--------+---------+----------+---------+--------+----------+ | Trial | Time | Type | Value | Region | Peril | Ints | Floats | +=========+========+========+=========+==========+=========+========+==========+ | 1 | 1 | Loss | 1 | FL | HU | 1 | 1 | +---------+--------+--------+---------+----------+---------+--------+----------+ | 1 | 2 | Loss | 2 | FL | HU | 2 | 2 | +---------+--------+--------+---------+----------+---------+--------+----------+ | 1 | 3 | Loss | 3 | SC | HU | 3 | 3 | +---------+--------+--------+---------+----------+---------+--------+----------+ | 1 | 4 | Loss | 4 | CA | EQ | 4 | 4 | +---------+--------+--------+---------+----------+---------+--------+----------+ | 2 | 1 | Loss | 1 | FL | HU | 5 | 5 | +---------+--------+--------+---------+----------+---------+--------+----------+ | 2 | 1 | Loss | 1 | CA | EQ | 6 | 6 | +---------+--------+--------+---------+----------+---------+--------+----------+ | 4 | 1 | Loss | 1 | FL | HU | 7 | 7 | +---------+--------+--------+---------+----------+---------+--------+----------+ | 4 | 2 | Loss | 2 | FL | HU | 8 | 8 | +---------+--------+--------+---------+----------+---------+--------+----------+ | 4 | 2 | Loss | 2 | FL | HU | 9 | 9 | +---------+--------+--------+---------+----------+---------+--------+----------+ | 5 | 3 | Fee | 3 | | | nan | nan | +---------+--------+--------+---------+----------+---------+--------+----------+ | 8 | 2 | Loss | 2 | FL | HU | 11 | 11 | +---------+--------+--------+---------+----------+---------+--------+----------+ Leger 2 (one.parquet): +---------+--------+--------+---------+ | Trial | Time | Type | Value | +=========+========+========+=========+ | 1 | 1 | Loss | 1 | +---------+--------+--------+---------+ | 2 | 1 | Loss | 1 | +---------+--------+--------+---------+ | 3 | 1 | Loss | 1 | +---------+--------+--------+---------+ | 4 | 1 | Loss | 1 | +---------+--------+--------+---------+ | 5 | 1 | Loss | 1 | +---------+--------+--------+---------+ | 6 | 1 | Loss | 1 | +---------+--------+--------+---------+ | 7 | 1 | Loss | 1 | +---------+--------+--------+---------+ | 8 | 1 | Loss | 1 | +---------+--------+--------+---------+ | 9 | 1 | Loss | 1 | +---------+--------+--------+---------+ | 10 | 1 | Loss | 1 | +---------+--------+--------+---------+ Output ledger: +---------+--------+--------+---------+ | Trial | Time | Type | Value | +=========+========+========+=========+ | 1 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ | 1 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ | 1 | 2 | Loss | -1.0 | +---------+--------+--------+---------+ | 1 | 3 | Loss | -1.5 | +---------+--------+--------+---------+ | 1 | 4 | Loss | -2.0 | +---------+--------+--------+---------+ | 2 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ | 2 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ | 2 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ | 3 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ | 4 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ | 4 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ | 4 | 2 | Loss | -1.0 | +---------+--------+--------+---------+ | 4 | 2 | Loss | -1.0 | +---------+--------+--------+---------+ | 5 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ | 5 | 3 | Fee | -1.5 | +---------+--------+--------+---------+ | 6 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ | 7 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ | 8 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ | 8 | 2 | Loss | -1.0 | +---------+--------+--------+---------+ | 9 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ | 10 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ Example notes: --------------- Scaled Load is defined in currency "DOUBLE" which means a 1 in the parquet files will result in a 0.5 in the output. For the ``currency`` parameter, a list of codes can be found at the following link; `currency code`_ . More details on ``currency`` can be found on our :doc:`/currency_exchange` page. .. _currency code: https://en.wikipedia.org/wiki/ISO_4217#Active_codes Example 2 --------- User uploads ledger with ``currency_column`` is set to "Currency", and ``factor`` is set to -1. Input ledger: +---------+--------+--------+---------+-----------+---------+--------+----------+ | Trial | Time | Type | Value | Currency | Peril | Ints | Floats | +=========+========+========+=========+===========+=========+========+==========+ | 1 | 1 | Loss | 1 | DOUBLE | HU | 1 | 1 | +---------+--------+--------+---------+-----------+---------+--------+----------+ | 1 | 2 | Loss | 2 | HALF | HU | 2 | 2 | +---------+--------+--------+---------+-----------+---------+--------+----------+ | 1 | 3 | Loss | 3 | USD | HU | 3 | 3 | +---------+--------+--------+---------+-----------+---------+--------+----------+ Output ledger: +---------+--------+--------+---------+ | Trial | Time | Type | Value | +=========+========+========+=========+ | 1 | 1 | Loss | -0.5 | +---------+--------+--------+---------+ | 1 | 2 | Loss | -4 | +---------+--------+--------+---------+ | 1 | 3 | Loss | -3 | +---------+--------+--------+---------+ Example notes: --------------- This example depends on DOUBLE, HALF, and USD having rates 0.5, 2 and 1 in the exchange rate table.