All posts by trader247

Speedy data

Speed is an important part of my bot development. When it comes to storing data, the options (data structures) I’ve been working with are array, collection and dictionary. When I Google for articles on speed, I get lots of information pointing to dictionaries as being the fastest for programming. But the main point of interest in the articles is the speed of looking up data. To find something in an array, all elements have to be looped through until it’s found (or isn’t). The use of keys in collections and dictionaries makes lookup faster as they can be targeted without looping, as long as you know the key(s). There are other advantages that make dictionaries preferred to collections, however they seem less important when looking at speed. A disadvantage of arrays comes when changing its size. Collections and dictionaries can have elements added without problems. To do the same with an array, we have to change its size to accommodate more data, which involves more time as the original is put into temporary memory and a new, re-sized, array created with the existing data then added to it.

This leaves me thinking that the two main speed disadvantages of arrays is searching for data and resizing. Here’s the question I’m looking to answer – if I know the size I want the array to be and I know the location of all the data held in it (so I can refer to each element directly) is there a speed benefit to using collections or dictionaries? Any help appreciated.

Comments

Steve commented –

The simple fact is you won’t win long term gambling or trading if your not getting value. With trading at least one side of your trade needs to be a value bet, optimally we’d have both lay and back as value bets but that’s harder to acheive automated than doing it manually. So whether you like it or not your monthly £26.55 winnings are coming from some of the value bets you place. It’s not particularily hard to identify where your value is coming from bets you’ve placed and from there you can start to fine tune bots so they work as efficiently as possible.

I did consider writing a much more in depth reply re:value and how to assess it but you do seem to be more interesting in cultivating some cut price Cassini blog personna than your bots for the time being.

Thanks for the comment.

Getting value when betting on an outcome means taking a better than true price. If you back at 4.0 but true price is 3.65 then you have value. If you lay the same at 3.2, you have value. Based on this, consider if I enter on this selection with a back at 3.2 and exit at 3.0, I have a green trade. My entry point bet is not at value with regards to event outcome. My exit point bet is at value with regards to event outcome. My trade as a whole has value as it results in a green book. This fits with the points made in the comment above but not with the statement “every bet sent should be allowed to stand on its own merits”. As described, my entry point, made prior to an exchange crash, is not at value in this instance – longterm losing (the argument here could be that the random nature of these crash events would, over time, likely have a neutral (less commission) effect on my bank but the infrequency would be more disruptive than that theoretical neutralness). So the term “value” is relative to the actions and results intended. (Other comment points noted.)

.
Tobias commented –

Hi, I am also into bot betting, but with a different twist compared to your approach… You seem to have something interesting going on, have you done some thinking about scaling up stakes? What kind of potential does your strategies have for bigger stakes? Anyway, god luck with your bots and keep us updated on your progress!

Thanks for the comment.

Stakes – I’ve allowed stakes to rise with bank in the past and found that return falls steadily to a point at which larger variance occurs followed by losses. I now have Oscar set to various stakes within a fixed range just below the point of where I saw more variance. I don’t intend increasing stakes on this strategy but may revisit this in the future, as things can change.

Updates- I’ve spent a lot of time in recent months on programming a new bot in visual basic, I’ll do a post on that soon (hopefully), it’s been a journey. 

.
On Twitter, Tony tweeted –

As much as I like the graphs and regular updates, no graphs and monthly rep = more dev time for you. Hope I get some subs money back! 😉

As mentioned, I’ve been developing a new bot, in a new to me language, so that’s where my time has gone. The graphs can be included in the blog, I can’t say I thought through that decision thoroughly. With a months worth of data on them I was thinking they may be less detailed, as in a line with less variance than reality, I’ll reassess. Subs will be discussed at the next AGM.

March ’17

Results

In a change to previous reporting I am moving from week/weeks to monthly stats. And no charts. Here are the results from March, beginning to end –

UK Dogs  Initial slow start to the month but then went on to a steady return. Overall good result with no changes to be made.

Markets = 1595
Bets = 10409
Volume =  £51949.79
Profit =  £59.65
Return =  0.115%

AUS Horses  The first half of the month saw profitable trading but the second half was all over the place with regular enough runs followed by sharp drops, streaks of losing markets, with no further profit added (this has been the same for the start of April). If this continues I will pause Aus trading.

Markets = 821
Bets = 6606
Volume = £62277.75
Profit = £26.55
Return = 0.043%

Comment
Steve commented –

The API crashing should have very little effect on a bots overall profitability other than the fact you’re missing out on opportunities when the site’s offline.

In a way, I agree. If I have an open position when the crash happens, I will either win or lose an unusually large amount for me. If this happens regularly, then, so long as I have enough cash in the bank, these wins and losses should roughly balance out. (This is stretching the view of chance and puts a lot of hope on a balance being seen across a relatively low number of events. I’d rather not rely on this to cover the effect of crashes.) Steve continues –

Every bet you place with a bot should be sent because you believe it to be value at that time and it should be allowed to stand on it’s own merits.

I think at this point Steve has missed my approach to the markets. With regards to the outcome of the event, I have no idea if my entry point is at “value”. This is because I have no interest or care of the event. It matters nothing to me if it’s dogs, horses, pigeons, camels, Pooh sticks or bottled messages that are racing. I am trading on the market movement, not the event. I believe my entry point has a statistical positive value if I can exit shortly after. Therefore, I never want a bet to stand “on it’s own merits” because it doesn’t have any merits (on it’s own). Look at it like this – I think the price is shortening and assume that there are willing backers and layers in the market. I effectively jump in between a backer and a layer, giving them both slightly poorer odds than they could have got and skimming a little bit for myself. That’s how I see it. Steve goes on –

I can understand the mentality of wanting a green book at the off but trading will always be easier to do manually rather than having some set time or ticks to balance your bets.

I disagree. Steve finishes with –

There’s a lot of easy money to be made botting don’t go wasting your time trying to tick for pennies.

In short – “trading will always be easier to do manually” even though “There’s a lot of easy money to be made botting“. With that logic any mouse clicking screen watcher should be raking it in. This I doubt. And the idea of  any easy money left on the exchange is one I don’t believe. But if you have found it, screw it for every last penny and don’t tell anyone.
 

Weeks ending 12-03-17

Well nobody spotted last weeks howler – I only titled it “Weeks ending 06-03-17”. I guess you did see it but found more amusement in keeping quiet. You are fun.

algotradingforfun added this comment-

Great 2nd week there. Need to think about handling the bf crash scenario when in autopilot. I don’t think it would be a disaster if not about but does create some extra risk.

Thanks. For me the crashes can be a bit annoying. Oscar backs first so the greatest loss is the stake, assuming a clean cut crash. If you’re laying first the exposed risk between entry and exit is far greater, add multi-runner trading and that increases, something to consider when setting up a bot.

 

Mike also commented-

The regular Betfair crash is a royal pain. Your take of their response is amusing and spot on. There is an API status page (not widely publicized) which is a little more real time than the “help” desk. Don’t know if you can link your bot to the status but might be an option. http://status.developer.betfair.com/

Thanks, again. A pain, agreed. I saw this status link on Twitter for the first time after this last crash and it does provide some confirmation but did seem a bit delayed. After I’d first seen the tweets I looked at the status and only one request was showing problems (/listmarketcatalogue maybe?) so trial and error would see if it could provide any bot use. But it was certainly ahead of the Saturday boy and his well thumbed guide.

 

One week on these charts. Interesting profile on the dogs, start flat, end flat, with sharp rise Friday/Saturday. All figures are in line with previous period which is good.

170312

Aus170312

Another milestone was passed with these results, I became eligible to pay premium charge as my lifetime percentage dropped just below 20 to 19.92%. I’d already used some of my allowance which I think was linked to data charges that are no longer used. So this week saw £1.98 taken off my allowance; at that rate it’ll be 9 years before I actually pay anything. Unfortunately, if my total charges percent continues to fall, the weekly PC will rise. A drop to 19.72% would have seen a PC of £5.50. This is the price of (small) success. On a positive note this does put me in a bracket with 0.5% of customers which, if Wikipedia can be believed, is either 20,000 or 5,500 people. What joy.

E-sports?

If you follow me on Twitter you may have seen my excited tweet on Sunday, showing off my win on the first e-sports market I’d looked at. (I used the method of laying a short priced favourite.)

esports win

But more importantly, it was the first e-sport event I watched. “What’s e-sport?” I hear you ask. It’s people playing computer games competitively. I watched the world championship grand final for the game Counter Strike: Global Offensive. There were two teams of five playing against each other in the final, having gone through qualifiers, rounds and knockouts. It’s more than just spotty teenagers in dark bedrooms. The top teams have dietitians, sports psychologists and fitness coaches. This event was held in a 11,000+ arena, streamed live online, with a prize pot of $650,000 ($250,000 for Counter Strike: GO). The world of e-sports is expected to grow rapidly in the coming years according to those in the know.

E-sports is something that as I’ve heard more about it, I’ve wanted to take a look. I’m glad I did as I did enjoy it. Whether I start playing remains to be seen. I follow Laurence Stanley on Twitter who blogs about his esport betting. His website is a good place to find out more.

For now the amount of money traded on the exchange was relatively low compared to other sports. There was a lot of money off the spread, with the spread being quite wide. There are a number of bookies offering E-sports betting, so I guess a lot of that on offer on the exchange was just waiting for takers so they could arb out with the bookies. As liquidity increases, a point will be reached where it takes off, bringing in even more money.

Weeks ending 06-03-17

Two weeks this period, starting from 20:30 on the 19th (see last update). A good return from both the dogs and horses but the first week ended barely up. The second week was one of the most profitable I’ve had.

170305Aus170305

Saturday 4th March saw another big Betfair crash* with the exchange offline for nearly an hour and betting disabled for some time after that. As I was at the computer when it happened, I took the opportunity to run updates on the VPS OS. I have it set up to tell me when updates are available but I choose when to install them. I’d advise any botter to do this, as the last thing you want is the computer restarting mid-trade after auto-updating. I also decided to add a bit of code to change how and when the bot saves it’s log sheet. A simple enough task as I’d already written the code for another, now retired, bot. Copy, paste, change sheet references and save location, job done, what could possibly go wrong? Not testing with live updating and thinking I know best and the bot locking up at 2am trying to repeatedly save a file that at the second attempt already exists, is what can go wrong. I’ve said it myself before now – always check, check again and test live. And check again. No harm done but missed most Aus horses on Sunday. Added to the missed Saturday dogs, the results are even better.

* All major exchange crashes seem to follow a similar pattern. Some people start reporting blips, pauses in the refresh rate. Betfair Customer Service (a questionable department title if ever there was) denies all knowledge with the stock phrase “It’s all good here”. Then comes total blackout. Many an unrepeatable turn of phrase screamed by the loyal customer base and the Betfair bods half acknowledge with the second phrase of the Betfair How to Keep Customers in Suspense Guide – “We’re looking into it”. Third line, after some threats of violence, use of very specific graphic language and calls for a mass exodus to Betdaq, is- “Apologies for this guys, our techs are on it”. Following a period of silence from the exchange masters, allowing for a build up of calls for refunds and shared stories of thousands lost, comes the market controlling monopoly confirming legal statement – “Refer to our Ts&Cs”. The first few markets after reboot are played cautiously before all but the over-exposed carry on as usual, allowing said monopoly off the recently polished hook.

A fellow PfB coder

Tony (@SportsBotter) commented –

Good to see you moving on to your own platform. I have had a go at it myself. At various stages I had to tick back what felt like most of the book. In a way making the mistakes forces you to understand how it all hangs together and what you need to change to implement your own strategies.

This is an important part of the learning process and can only be gained by doing the coding yourself. For those who just want to copy and paste the program, the ability to make it their own is reduced. As Tony points out, by making mistakes and then trying to find where you’ve gone wrong, you start to see how the modules come together, where variables get their data and then where and how that data is used/manipulated. Trying to skip this exercise will only make any future changes more difficult.

Now I am quite pleased that I can offer bets and cancel them just using basic non strategic triggers plus change the markets and periods that interest me etc. Quite confident I can now push on from this point with some trading techniques that I couldn’t be bothered to actually perform myself on a day today basis. Now if I can do it …with my reputation… then anyone can ;) 

The video shows an example of what Tony has created, something that fits his own criteria, using the Programming for Betfair code. He also has the Betfair site open so you can see the bets in the markets. Getting to this stage , with a working app, does give you confidence. I am now coding and testing my own bot following the same process that Tony has.