TWM Wiki
Market Data – Manage Real-Time & Historical Trading Data
Understanding Market Data
To get market data it is fundamental you have internet connection.
If market data has previously been downloaded, users are able to perform tests using stored market data, which will be sourced from the default Documents folder and then from "\Twm\DB\HistoricalData\ folder". For example:
Each Exchange is assigned a folder and data is stored inside, in accordance with instrument types (eg. spot, futures), and time frames. TWM sources data from the provider as it is.
In order to download new market data a connection has to be set up to the appropriate provider. Please address the connection section on more information about setting up connections.
Data Series Window
When opening a new chart, users need to select the data source from the data series window.
The data series window, allows users to select the data souce from the chart they are about to open. This window allows users to select which data formats to be sourced from the server, and different data formats can be differ depending on the provider.
The instrument selector allows users to select the instrument symbol from the users custom instrument list or from the default list provided by TWM. User custom instrument lists will appear below the default one once created. For more information on how to create users instrument lists refer to the instrument section of this manual.
Data Load Options
Users are able to load data using two available methods:
- Days to Load: based on how many days an user wants to load, days will end on current date
- Custom Range: dates that go from start to end.
Loading Algorithm
The first time an user loads data, TWM will automatically attempt to download the maximum available period from the server, no matter if the user asked for a certain period. Please make sure you have a stable internet connection, as it might take some time to load. Subsequent calls for data, will make download to TWM only missing data. Please allow for longer waiting times upon your initial data request, especially if its small data formats, such as 1 min.
Data Storage
Historical price data is stored in individual instrument folders, in CVS files at the following directory:
Furthermore, TWM also maintains metadata for every downloaded series. You can review it in Tools → Options → Historical Data.
Managing stored data
The Historical Data dialog lists all instruments and time‑frames currently on disk. Users can use the context menu (right‑click the root folder in the file tree) to:
- Add – to open the data‑series dialog and download new data.
- Remove – to delete the selected instrument’s files and clear its metadata.
- Remove All – to wipe the entire Historical Data directory and remove all metadata.
Right click on root folder name, in file tree to view the context menu with add/remove options
Other Data Calls
Additionally data can be sourced by TWM when using specific sections such as validator and optimizer. Please addres these sections for more information on when and under what circumstances data will be sourced from server whilst using these sections of TWM.
Important Information
If you attempt to delete data manually from the disk, directly from the folder, the metadata will not be deleted from the database, causing a desynchronization and a corrupted scenario.
If you accidentaly run into this situation, you will need to close TWM and delete the database file completely, which is located at:
Furthemore you will also have to delete all existing data. After you restart TWM the database will be recreated, and no metadata will be present in the database. If you need to delete data, please use the option menu to delete it in synchronization with the metadata.