Given the amount of data considered, and the number of iterations needed in order to complete the task, it was clear that a cloud-based approach was needed. Aikyo created a complex pipeline in 4 steps. First saving all the raw data in an Amazon S3 bucket, using geography and dates as keys and subfolders. The second step was to process raw text files, filter the data, and denormalize them into the AWS NoSQL database ( Dynamo DB ) to allow strategic querying of the data. The third step was to compute mono underlying strategies and a wide array of signals. The final step was to compute multi-underlying arbitrage strategies by using signals for picking the most optimal options at each draft.
