Speedy data

Speed is an important part of my bot development. When it comes to storing data, the options (data structures) I’ve been working with are array, collection and dictionary. When I Google for articles on speed, I get lots of information pointing to dictionaries as being the fastest for programming. But the main point of interest in the articles is the speed of looking up data. To find something in an array, all elements have to be looped through until it’s found (or isn’t). The use of keys in collections and dictionaries makes lookup faster as they can be targeted without looping, as long as you know the key(s). There are other advantages that make dictionaries preferred to collections, however they seem less important when looking at speed. A disadvantage of arrays comes when changing its size. Collections and dictionaries can have elements added without problems. To do the same with an array, we have to change its size to accommodate more data, which involves more time as the original is put into temporary memory and a new, re-sized, array created with the existing data then added to it.

This leaves me thinking that the two main speed disadvantages of arrays is searching for data and resizing. Here’s the question I’m looking to answer – if I know the size I want the array to be and I know the location of all the data held in it (so I can refer to each element directly) is there a speed benefit to using collections or dictionaries? Any help appreciated.

Advertisements

3 thoughts on “Speedy data”

  1. Where do you host your programs? Cloud? You streaming? I think I/O and network latency combined with serialization are likely to be your biggest bottle necks. With python you can use cprofile to see what is slowing you down not sure what the VBA equivalent is.

    Like

    1. VBA and Visual Basic are two different entities. Trader247 is using Visual Basic to create his bot.

      He poses an interesting question the answer to which is somewhere at the back of my mind and will need coaxing out over the weekend.

      A few questions. Why do you want to use an array? I think you have answered the speed benefits yourself but I can clarify.

      BPT – http://www.betfairprotrader.co.uk

      Like

  2. >>> To find something in an array, all elements have to be looped through until it’s found (or isn’t).

    You probably know this anyway, but…

    If you use a List object to hold your collection of objects/things, the .Find() method of this class returns the first occurrence of the object or item you’re searching for – I haven’t tested it for performance but I think it will be faster than searching through every element of an array. There are other similar methods along the same lines as well (eg. FindAll, etc.).

    It also has a BinarySearch method you can use on sorted lists.

    Plus you can serialise/persist it very easily as well. I use it all of the time.

    I use C#, but here’s a link to the VB page from MSDN (VB looks very verbose and noisy compared to C/C++/C#/Java/etc…) 🙂

    https://msdn.microsoft.com/en-us/library/x0b5b5bc(v=vs.110).aspx?cs-save-lang=1&cs-lang=vb#code-snippet-1

    Cheers,

    Ken

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s