Simian Savants LLC

        Charting Software LogoSSChart Candlestick Stock

        Charting Software banner
 IB TWS - Setup & Usage  IB TWS Socket Interface  
 

 SSChart setup when using IB TWS for data

When you first install SSChart, it is not configured to get data from IB TWS. You must follow the instructions on this page in order to connect and get quotes from IB.

We recommend using the installed version of TWS rather than the browser version. (We are not testing the browser version). We update these pages with new info regularly. Check the last revised date at the bottom of each page, especially after a new release.

Note: After the trial expires, you will need either the combined monthly lease, OR, the fixed lease plus permanent license to continue to interface with TWS. See details here.

IB TWS Socket Interface

There is now a simpler built-in direct socket connection to IB TWS in SSChart. The older separate router program (SSQrouterIB.exe) is now obsolete and not currently supported. You do not need to install the API from IB, just the latest TWS version. (This is still considered a "router" for licensing purposes.)

IB TWS Integration

After you have completed the installation of SSChart follow the instructions here for configuring it to get data from your IB account through the TWS API.

 

To Configure SSChart to get data from IB TWS

  1. Start TWS.
  2. Go to the TWS {Configure | API} menu and verify that the box next to "Enable ActiveX and Stocket Clients" is checked.
  3. Confirm that {Configure | API | socket port...} setting in TWS is 7496, if not, set it to 7496.
  4. Start SSChart.
  5. Under the { File | Preferences } menu item,
    1. on the Data tab enter the Hostname (computer name) of the system on which TWS is running. (If SSChart and the TWS are run on the same machine, you can use "127.0.0.1" in the hostname field instead of the computer name, this setting applies both to the IB TWS Socket Interface and the SS IB Quote Router).
    2. If you will be charting data outside regular US hours (like futures) also make the Keep Extended Market Hour Data setting discussed below.
    3. Switch off Auto Collect
    4. on the Misc tab under IB TWS Options, set Client ID to 0, and Port Number to 7496. (Do NOT check the Auto Start TWS box until you have successfully connected without it and understand the usage and limitations discussed below. Filling in the Account field with the account number as shown in the banner line of TWS will allow autostart to succeed when more than one instance of TWS is run (connected to different accounts). You may leave this blank if you only have one IB account, and you have not modifed the banner setting under TWS configuration. The important point is that this needs to match part of what gets displayed on the TWS banner line. The Advisor Account box should only be checked if you are connecting to an advisor account (see this). The Inhibit MktData Request field is used to inhibit the request of data ticks for various types of contracts that you may have open positions or orders for. Otherwise these will cause the request of market data for each which might put you over your symbol allotment if you have a large number of such orders/positions.
    5. <click> [OK]
  6. Under the { Data | Internet Sources... } menu item, in the Quote Data field, select IB TWS Socket Interface, and make other selections as shown below, then <click> [OK].
       NOTE: Do not type in these fields, <click> the down arrow button by the field and select from the pull-down llist.
  7. Exit SSChart and restart (this will save your settings made above).
  8. Create at least one intraday chart with a symbol you want data for. (If you want data outside US stock trading market hours, like futures, you must <right click> in the chart and select Show All Sessions or you will only see the normal 0930-1600 hours. Note when Show All Sessions is selected it will show a check mark beside it).
  9. Make sure TWS is running.
  10. Start collecting quotes by selecting { Data | Get Continuous Quotes } or <click> on the quote collection button on the toolbar.
  11. Under normal circumstances, SSChart will connect to TWS and automatically answer the "Accept incoming connection?" prompt by TWS. The quickest way to verify you are getting data is to create a Time&Sales window and drag-and-drop a symbol into it. This will print every trade reported by TWS.

Restart procedure:

Once you've successfully complete the setup above and are getting quotes, the restart procedure is simpler. Whenever you exit SSChart normally (didn't crash or kill it with taskmanger), the configuration and layout are saved. Restart should only require the following steps:

  1. Start TWS
  2. Start SSChart
  3. Switch on continuous quotes (toolbar button, or {Data | Get Continuous Quotes} ).

You can use the auto collect feature under {File | Preferences} to turn on quotes, but you must insure that TWS is running when the quotes are switched on. You can do this with some limitations using the Auto -start TWS feature described below:

 

Symbol encoding:

To get futures, options, or indices data you need to use special encoding of the symbol name (in a chart or slf file).

Data collection times:

The normal data collection time is set for the US equity markets normal hours. This is 0930 - 1600 eastern time, in order to keep the data for any time outside this (futures, foreign markets, extended market hour equities, etc.) it is necessary the check the "Keep Extended Market Hour Data" box under the user preferences data tab under { File | Preferences... }. See example below:

This will cause the data outside of normal market hours to be saved in the .1min file for each contract, however to display it in an intraday chart, you will also need to <right click> in each intraday chart and select Show All Sessions in the popup menu in order to see any data outside of regular session hours.

Additional TWS configuration for trading

If you are using the trading or graphical trading interfaces of SSChart you should make the following additional configuration settings in TWS:

Under the TWS {Configure | API} menus check Download Open Orders on Connection and Fire openOrder on status' change.

Under {Configure | Misc} check Create Audit Trail. This will keep a log of all order actions sent from TWS either directly or via SSChart. You can view this log under the TWS {View} menu.

You need to be able to use TWS to send, modify, or change orders in the event that there is a malfunction or mistake through the API interface. Do not use our frontend as an excuse for not learning to properly use TWS. A couple of years ago IB had default order hotkeys setup when you installed it. You now need to do this yourself via the TWS {Configure | Hotkeys} menu. Here are some of their old recommended defaults:

 Key  Function
 T  Transmit
 C  Cancel
 B  Buy Limit at bid
 S  Sell Limit at ask
 U  Increase limit price 1 tick
 <shift>U  Increase limit price 4 ticks
 D Decrease limit price 1 tick
 <shift>D  Decrease limit price 4 ticks

The price up/down hotkeys work a lot quicker than trying to use the mouse and the pulldown selection. Another very useful key to have setup is the "Close position" key which sets up an order to close your current position". You can configure any of these to transmit immediately if you wish, but I prefer to double check the parameters first then hit "T" to transmit.

If this is all new to you, you should really be learning how to set this up and use it in TWS before you use our program to trade (or any other front end).

You should also use TWS's {Configure | View | Layout Manager...} to configure the TWS order line to show all pertinent data, including "OCA Group", "Position", etc.

Auto starting TWS

SSChart can now auto-start TWS and login if it is not already running when quotes are switched on. (This is controlled by the checkbox under the {File | Preferences...} Misc tab.)There are a number of things that can prevent this from working, including:

  • Running more than one copy of TWS at a time
  • Mouse and/or keyboard activity on your part while the program is attempting to log into TWS.
  • TWS unable to connect to IB servers at the time the attempt is made

Any of these things may result in the auto-start and login not succeeding. Auto starting from another application is not something built into the TWS API, but rather we (and anyone else doing it) must resort to some messy programming tricks to accomplish it. Never the less, with a little care it can succeed in allowing the program to restart TWS and collect quotes again after the IB server down period which happens around midnight eastern time.

To fully automate this you should also use the {File | Preferences...} Auto Run tab settings to Automatically Collect Quotes.

IB's servers now are always on except for about 1 minute when they reset sometime between midnight and 3AM eastern time. The times shown in this dialog are in local time (Pacific time for us). We use these settings, stopping quote collection at 11:50PM (23:50), and set TWS exit time to 11:51PM. It can take a minute or two to shut down, so we allow it at least two minutes. Then restart quotes at 23:53 (11:53PM) which autostarts TWS again.

If the IB servers are not actually up when you attempt to start the process may fail requring you to manually restart quotes.

If you have trouble reliably restarting this way, try a different shutdown time. If IB servers are not responding when you try to restart we have seen times when TWS won't properly log in until you stop the current attempt and restart TWS. If you need to restart TWS, its a good idea to restart SSChart also just to make sure there is no thread still attempting communication on the old socket. (We rarely ever reboot our computer) We set TWS auto-logoff time (under {Configure | Misc | Auto logoff time} in TWS) to1 minute after we stop quotes in SSChart.

Even when this succeeds, we recommend you exit and restart SSChart at least once a day when ever possible as this cleans up internal memory allocations.

Notes: Using the auto-start TWS feature poses a security risk. If anyone besides yourself has access to your computer, they could gain access to your TWS account by starting SSChart. If this may be a problem, do not configure SSChart to auto start TWS.

If you are "participating" in the IB security program you will need to complete the login to TWS manually each time by entering the security code, from your card or security device. This make unattended login impossible.

We have found that sometimes when we have been running TWS all night long, that the next morning it does not retrieve historical (backfill) data when you used the {Update} function. In this case it is necessary to shut down TWS and restart it, it then seems to work fine.

SSChart has a sound that will be played when auto-login of TWS is initiated if you enable it under {File | Preferences} on the Sound tab. This helps alert you, so you can complete the login with security code manually.


Data Limitations

TWS now will supply up to 1 year of intraday backfill. To use this, <right-click> in a chart when quotes are on, (and you are connected to TWS) and select {Update}, the chart should be updated with the backfill within a few seconds. SSChart will attempt backfill for as far back as the "days to keep" setting under {File | Preferences...} indicates.

The streaming data from IB does not include a timestamp on the trades. We assume each trade comes through in realtime and use the current time. This is generally correct, but not always. When you first start collecting data for a symbol, the first value is the last trade that occurred. It could be an hour ago, and we have no way of knowing, so when you first startup, the first point is not accurate in time. If you shut down SSChart and restart, the first point may be a duplicate of the last one you had with a new fictituous trade time (the current time) if no trades happened during the time you were shut down. When trades happen rapidly, TWS does not send every trade, so tick count and number of bars in tick charts will be incorrect. Any time TWS loses connection or restarts, we miss data, you need to do the update procedure to fill in the gaps. The data will only be as accurate as what TWS is receiving.

The day's open, is not present in the IB data stream, so we use the first price we see after opening time.

Ticker (symbol) limits

TWS normally limits the number of symbols you can get data for to 100. (If you generate a very high amount of commissions in a month, they increase this by some formula disclosed on their website). If you have open orders in TWS (and are operating with client ID 0) TWS sends the order information to SSChart, and SSChart will request data for those symbols if it has not already done so. In some case, like when you are trading options, and single stock futures, you may not want to use up your ticker limits by requesting data for these in SSChart. The check boxes on the "Misc" tab under {File | Preferences...} allow you to exclude these security types from data collection (in SSChart).

You can further control this by creating two possible files in the CONFIG directory. Each file should be text format (use notepad.exe) with one symbol per line.

Any symbol listed in CONFIG\INHIBIT.TXT will not have data requested, overriding any other rules.

Placing a symbol in CONFIG\ALLOW.TXT will override any other inhibit rules (like the check boxes under preferences mentioned earlier), and data WILL be requested.

A new file has been added in version 2.96. The file CONFIG\SymbolMods.txt will allow overriding the <currency> and <exchange> values used when requesting a stock symbol. A sample file is in the SSCHART\USR directory. If you wish to use this feature copy the file to the CONFIG directory and edit it. (Comments in the sample file will explain the format). You can use this feature to force simple format symbols to request data from non-US exchange with non-USD currency. This requires an entry in this file for each symbol so used. The encoded forms shown on the symbol format page work as normal without any entry in this file.

Backfill from TWS

Backfill is available whenever you are connected to TWS (quotes on). To obtain intraday backfill, <right-click> in any intraday time based chart (5-Sec ... 60-min), and select {Update}. This will update the data you already have. If you select {Reload} instead, your existing data is discarded before the backfill is done, so you will have ONLY what was supplied by TWS. The backfill from TWS can be quite slow in coming, we now do this in a separate thread so as to not delay plotting of realtime data. While the update is in progress for a chart the word "updating..." will be displayed on the line with the current quote. When this disappears the update of the most recent day is finished and displayed, updating of data for days farther back continues in the background and will be merged into the chart once the process is completed. If it fails for some reason and message usally pops up in the status messages window. The SSChart status line (at the bottom of the window) displays a message for each block of data requested in the background.

If you try getting backfill several times and none is available. Try shutting down TWS and restarting it. We've found this usually fixes this problem. We usually run our program 24-hours a day, and find after being on all night, we need to restart TWS to get the backfill. It should only take a few seconds to get the backfill data when it is working.

You can update every symbol in and SLFMap in the background by selecting { Update All Intraday } from the SLMap pop-menu. (When you <right-click> in and SLFMap). A small blue square is shown in the lower left hand corner of each symbol cell in the SLFMap while the update is pending. Once this square disappears the update is complete.

We have recently optimized the IB backfill to give the best performance available within the throttling restrictions imposed by IB servers. (Their servers limit the frequency of backfill requests to keep them from being overloaded for trading and quote purposes, which have higher priority). IB does impose a restriction, if you repeat the identical data request within 15 seconds, it causes a "pacing violation", and data will not be returned. The other restrictions we avoid automatically in the program.

Data for seconds charts (5-SEC, 15-SEC, or 30-SEC) is not saved to disk, conseqently when you open a new chart for a symbol it is blank. If you change symbols in a chart the data for the previous symbol is discarded and the new symbol will show a blank chart. To backfill the chart, select { Update } from the chart pop-up menu. (<right-click> in the chart to get the pop-up menu.)

The recent version of TWS can now supply data up to 1 year back. . In order to update from our cme-futures download files, you will need to switch data collection off temporarily, otherwise the update will come from IB's backfill instead. (Note: Since CME stopped supplying this data for free as of Nov 1 2007, this is not longer available. We do still have several years of tick data on CME futures available that can be used with the playback feature up through that date).

Volume

The TWS data stream does not give every sale when trading occurs rapidly, they do give every price point. It is throttled (apparently) to some max data rate to keep the price display from getting behind in times of rapid trading. The data stream was optimized for updating the TWS quote display and only transmits changed fields which means when two consecutive sales have the same price and size, there is no change and therefore no transmission! Assuming their total daily volume transmission is accurate we can deduce additional sales (including consecutive sales with same price and size) and get an accurate picture of volume for stocks, although some individual sales may be aggregated. All of this applies to time&sales display also.

IB now seems to transmit volume for Globex futures also, but as of last check did not transmit it for the Hang Seng index future on HKFE. If you are trading other markets you may want to check if you can get the volume field in TWS. If you do not get it this implies that volume info will be less accurate (since we will not be able to get trades with same size and price as the previous trade). To see volume in TWS you have to enable it's display in the page layout. Whether or not you enable it in TWS doesn't make a difference in what SSChart gets, SSChart will get it, if it is in the datafeed).

Comparison of the e-mini time&sales volume from CME with the volume reported by IB, shows that IB apparently does send correct total volume for this contract, but individual trades may get aggregated to throttle down the data transmission rate.

The tick stream for stocks appears to be missing a lot of trading volume, compared to that shown by the backfill.

Further info

[Back to Top]

 
Last Revised Saturday, 03-Apr-2010 22:54:38 EDT
SSChart is a trademark of Simian Savants LLC
Copyright (c) 1999-2009 Simian Savants LLC. All rights reserved.