Tag Archives: Charts

August ’17

The streaming version of Gruss is better now, none of the issues of late with missed market list updates or getting stuck on expired markets.

The new Oscar on the Aus horses has performed flawlessly (apart from another schoolboy error that lead to it trading one race the first day out and not moving on). I now need to add all those changes to the UK dogs, not that I have any trouble there but the code is neater and more efficient, so good practice to implement.

I haven’t done any more with the bots other than update Gruss when new beta versions are released, as I’ve been busy with other things.

A good chart from the UK dogs, less up and down than last month.

chart_ukdogs170831

Generally good for the Aus horses. The sharp up then down was a run of four races where only one back bet was placed. No offset or green bets placed/matched. I’m assuming there was either an API issue or connection problem. These races don’t seem to suffer from early starts, so it isn’t that. And they were different venues. One of those things.

The sharp drop prior to those four errors was a failed offset/greening sequence. There were 7 back bets placed/matched fully but the seventh lay bet wasn’t matched and then greening only partially matched.

These were all before the new code for greening went in, so I’ll have to wait and see if it happens again. I’ve probably said it before but in the long run, these errors tend to have very little effect.

chart_aus_horse170831

Although I praised the US horses last month, compared to the Aus dogs, this chart has pushed me to stopping the bot. Four missed offset bets resulting in a gain overall. I think the way my algo trades the markets, it just isn’t worth the risk with too few bets placed. There isn’t the turnover to recover in time for the next problem/error.

chart_ushorse170831

As mentioned last month I went to the Ebor festival at York for the Saturday races. It was a good event, nice weather, plenty to eat and drink. I’d gone with a tight limit on the betting and placed bets with the on-site bookies, to enjoy the experience. I don’t really like gambling though (in the traditional way), I can’t get past thinking I’m just going to lose as I don’t know anything about picking winners. Won nothing, surprise, but had loads of laughs and enjoyed getting to the front to see the horses race past the finish post, great atmosphere and very impressive. And Saturday night in York was a laugh too, finished with a kebab, “large… hic… everything on it mate… hic…  “, which seems like the best thing ever at the time but probably best forgetting about afterwards, rather than contemplating it’s content.

Advertisements

July ’17

Liam commented on Streaming and NSW

Its important to note that latency from streaming is a function of how quickly you can process the updates. If your CPU is unable to process the updates quick enough betfair will apply conflation which is what you are probably seeing. Of course network latency can impact this but its is more likely to be due to CPU / algorithm you are using to process the stream.

“””
Writes to your connection are directly effected by how quickly you consume data & clear your socket’s buffer
Consuming data slowly is effectively identical to setting conflation.
If you receive conf=true flag on a market – then you are consuming data slower than the rate of deliver
“””

Thanks for the comment.

This is something new to me and trying to find further info doesn’t reveal much. I haven’t seen any reference to it in the Gruss error log. Although I’m not entirely sure I understand this correctly. The statement in the quotes talks of “effectively setting conflation” by consuming data slowly. But who sets conflation? Which I guess to mean that updates are delivered in groups, merged, rather than individually. Is it something you can do from this side? Or is it a decision made by Betfair? But how would they know how you’re processing the data? My understanding of streaming is that it arrives at your computer as is, stored in a local cache(?), then processed by the program/bot. Would it be delivered/streamed to the computer conflated? Or conflated in the local cache for processing? I don’t know but it sounds interesting and any more information would be appreciated.

I’m only using Gruss at the moment as I’m not really excited by the prospect of coding for streaming in VB. My ventures into VB continue but not with Betfair. I finished a basic lay stake calculator and am now on with a more advanced version, trying to get a rolling list to display neatly. Also tested some scraping code and played with excel controls. I find Visual Studio so much more advanced than the VBA IDE that coding/debugging is a lot clearer and faster.

Back to trading. I’ve two test bots in coding at the moment. One is a reworking of a bot I’ve trialled on and off. Simply, it scrapes data and then trades. The thing is, each time I go back to it I have to change a chunk of code as the websites, as is always the case, alter their layout or swap some data from html to a type that doesn’t scrape easily (javasript/flash?). On the bright side it does mean that I learn some new code or technique.

The second test bot is an update to Oscar to take in the recent changes introduced with streaming. There is two parts to this – market navigation and greening. I’m having problems with the bot stopping on suspended/closed/abandoned events and have missed a number of days across the different bots. Although some change was made by the Gruss team, improving the situation, it is still happening. So to help from my end, I’m changing the way my bots refresh the markets (list of available events for each bot) which are held in the Quick Pick List. This will also include a change to how the bots jump from one market to the next; switching off full stream, which forces a specific refresh rate, prior to leaving an event, then switching full stream on once a new market is loaded and certain conditions are met.

EDIT: Team Gruss have said they’re on with fixing this problem 🙂

A new feature now available is the eagerly awaited and much anticipated greening trigger (it really has been wanted for ages and requested loads on the forum). My greening code is quite messy with multiple attempts to green. It’s been a crude hack from the early day’s that I’ve bodged to work rather than start fresh and I’m happy to see it go to be replaced by a simpler solution.

Oscar now and the UK dogs. Hmm. It is positive, that’s about it.

chart_ukdogs170731

Aus horses. Also positive…

chart_aus_horse170731

Aus dogs. The story of the 6p I wish I still had. Seriously though, there just isn’t the liquidity so the result could be anywhere to be fair. I stopped this bot at the beginning of August with a profit of 12p since May. I don’t see any reason to risk a big loss on these markets where it is unlikely to ever make it back.

chart_aus_dogs170731

Us horses have produced again with 0.121% for July. June was 0.118%. Not vastly more markets traded than the Aus dogs but there are races that get as much money as the bigger UK dogs races and the higher liquidity gives more opportunities to trade. I guess these races are the televised ones?

chart_ushorse170731

I’ve been invited along to the Ebor festival at York races on the Saturday. I’m taking a bit more out of the bank for spend. It’ll be the first big race I’ve been to having only been to the races three times before – regular events at Doncaster, Ripon and Pontefract – so am looking forward to it.

June ’17

The big event of June was the release of a beta streaming version of Gruss. As it is in development, the Gruss guys requested feedback and there were a number of bugs highlighted by users. We are now on the sixth release, which is good to know the program is being worked on and improved. The bug that I found related to moving on from suspended markets but after I fed back via the forum, a fix was quickly released. It took a little time for me to grasp the full effects of streaming. At first I thought the refresh was poor as updates were very random. The stream only updates when there is something to update, ie market activity, and doesn’t waste bandwidth by refreshing the same data repeatedly, as before streaming. This gives a refresh chart that can have quite large gaps between updates, especially on markets with some time to go before off. Now, with only a few minutes to off, the refreshes come in more than every 200ms. As there are no requests for price data, there’s no added delay. To note, the charts on the VPS are showing lower times than those at home. The request delay will still be relevant when placing orders in the market but no data is available on what it is.

Another major event, for me, was the changes to the Aus turnover eligibility. I posted about this here. I’m on with the coding around this. I have a section of code that only runs once when a market is selected and then isn’t run every refresh. I’m adding the NSW code there, which was straight forward for checking against the list of courses but tracking the traded back bets over a week is a little more complicated.

The UK dogs have done good this month. I’m considering adjusting the stake range to allow higher bets. I want to trial it on specific markets first. I’m thinking of those that are televised and tend to have much higher activity.

chart_ukdogs170630

An improved chart from the Aus horses compared to recent months. It’s nice to see a good return as I was beginning to lose patience with it and was considering stopping this bot. It’s been a long time since I ran Oscar on the UK horses as there was no value in it for me and I was thinking the Aus horses were going the same way. They still might, to be fair. But for now, with this, it will continue.

chart_aus_horse170630

US horses – better result than not trading them at all. There are some really well funded races that I’m missing, purely down to start times been well off. I’ll continue moaning about this point until I finally get a solution in place (I’ve had some good suggestions from you but the code don’t write itself, I should get on with it).

chart_ushorse170630

Definitely lower activity on the dish-lickers. An unfortunate loss keeping the return in the negative. Not much harm in continuing for now, I consider this my experimental contribution (why not?)

chart_aus_dogs170630

May ’17 – and the art of separation.

Work on the VB bot was frustrating me so I decided to pause it and have a play with Oscar, my VBA bot. I’ve nearly always run one instance of Oscar, navigating between different markets and sports based on some preset criteria. I decided to split the sports, running an instance for UK dogs and one for Aus horsies. This has the benefit of not missing conflicting events across the two sports. The reason Oscar didn’t do this originally is because back in the day, Betfair charged for making excessive calls above a relatively low amount. This changed some time back but I hadn’t.

This new set-up runs well and so I added Aus dogs, also running in its own instance. And why not US horses? OK, they are now covered in another instance. (Previous attempts at US horses had not seen many trades but most races were missed in favour of the other markets.)

I’ve monitored some cross-over times on the VPS and I haven’t seen any drop in performance. At some points in the morning, three of the bots are running at 0.2s refresh rate but I’m still getting a delay of less than 20ms on each.

I did notice the other day that the US horses were buggered by some error in the stated off time. For one venue the times were in the quick pick list but when the markets were selected, the off time was around an hour and a half out. This may have been an API issue but if I see it again I’ll look at coding to handle the mismatch between the two.

Next, Chris commented on Speedy data 2  –

Very interesting articles about bot speeds. I have been looking into the same. I have looked at my algorithms and have improved them. They now return values within 3-4 ms. However the main bottle neck is the price refreshes. Without streaming they currently have a price refresh at 200ms, my prices can be 180 ms out of date. If I was able to implement streaming I could improve my robots speed by a huge amount (probably 100ms), dwarfing any gains that could be made over optimising my robots. So I would suggest that the bottleneck is in your price refreshing and you could see a large improvement with your bots if you were able to stream prices.

Thanks for the comment. 3-4ms is fast and I haven’t seen those speeds from my bots yet. How are you timing the code? And what language are you coding in? The arrival of streaming made me less eager to push on with the VB bot as I don’t want to put all the time in to get the code perfect just to see it become old-hat overnight. Gruss, the software I use for my VBA bots, are releasing a beta streaming version soon. When I’ve had a go with that, I’ll look at how to stream with VB. There’s no point at all in not streaming if it’s faster and as reliable (collective eye roll) as the API-NG.

And now, some charts.

UK dogs have done ok, nice steady performance.

chart_ukdogs170531

Aus horses continue to throw some bad results. The three sharp drops in this chart have different causes. The first is actually 3 losing markets together, so no problem there. The second was an error, a problem I haven’t seen for a while where an extra lay is submitted for some reason. I’ve previously thought this is down to timing and the bot missing signals at specific points, eg when greening occurs and a bet is taken at the same time as a CANCEL-ALL command is triggered. The third was just a bad run of multiple bets being placed within the stoploss window, all eventually losing trades. When I’ve attempted to overcome this particular event in the past, the number of trades significantly reduced. I may look at this again, specifically in the Aus horse markets, but with a more complex solution.

chart_aus_horse170531

NEW – Aus dogs, although not many markets, has a good looking chart, certainly one to watch. Stakes are still hovering around £2 for now.

chart_aus_dogs170531

NEW – US horses, only 3 days here so wait and see what happens in June. Interesting to see average bets per market at 9.9, with other sports being

UK dogs = 5.6

Aus horses = 6.1

Aus dogs = 2.9

chart_ushorse170531

You can find all of Oscar’s UK dogs charts on a single page now – see here.

 

April ’17, with charts. And some charts for March.

Charts are back. First up are the charts for the period 13th to 31st March. The figures won’t match with the March post for obvious reasons but will follow on from the last P&L charts.

170331Aus170331

On to April. The dogs had an ok month overall but the profit mainly came in the first half and then not much after. No changes were made so not sure why it happened. Maybe the strategy is losing its edge.

170430

The Aus horses have not done well. I stopped trading on the 29th with a view to resetting some limits. It took me 5 days to actually sit down and do something. Minor changes made to the limits and the stake was reduced by 75%ish. Up to yesterday not much has improved.

Aus170429

I haven’t done as much programming as I’d like recently but the garden is coming on and I’ve done a few guvvy jobs to help pay for it, so not much spare time. I’ve had a few comments on here and Twitter about speed which I will put into a separate post when I get time. The latest one from n00bmind was detailed and worth a read itself.

In the meantime I’ve added an Oscar clone to the VPS to have a go at Aus dogs. No idea what liquidity is like but we will see.

A chart, I see

I’ll return to the speed question soon (as I have an idea for a testing bot, just needs writing), the results of the little testing I’ve done are not really good, as in they don’t really move me forward. Development of the VB bot is still progressing though, well my programming ability is, with the more I learn I add/change what I want to do but it’s all good. I’ve been playing with what data to collect/monitor and how to handle it (to avoid unnecessary bloating of the bot). As I like to visualise things, I’ve been presenting data in different ways. Below is one of the charts I created, just with excel, to show matched volumes. It covers the second favourite for the final two minutes before the off in a middle-of-the-road greyhounds race. The data is at 1 second intervals with a 4% decay added to project but not obscure the matching. The price can be seen to rise as it approaches off time (front of image) with last-price-traded at 4.0. It isn’t much use on its own but I like how it looks. Adding another parameter, or two, gives more meaning.

match_vol_dec_chart

Weeks ending 12-03-17

Well nobody spotted last weeks howler – I only titled it “Weeks ending 06-03-17”. I guess you did see it but found more amusement in keeping quiet. You are fun.

algotradingforfun added this comment-

Great 2nd week there. Need to think about handling the bf crash scenario when in autopilot. I don’t think it would be a disaster if not about but does create some extra risk.

Thanks. For me the crashes can be a bit annoying. Oscar backs first so the greatest loss is the stake, assuming a clean cut crash. If you’re laying first the exposed risk between entry and exit is far greater, add multi-runner trading and that increases, something to consider when setting up a bot.

 

Mike also commented-

The regular Betfair crash is a royal pain. Your take of their response is amusing and spot on. There is an API status page (not widely publicized) which is a little more real time than the “help” desk. Don’t know if you can link your bot to the status but might be an option. http://status.developer.betfair.com/

Thanks, again. A pain, agreed. I saw this status link on Twitter for the first time after this last crash and it does provide some confirmation but did seem a bit delayed. After I’d first seen the tweets I looked at the status and only one request was showing problems (/listmarketcatalogue maybe?) so trial and error would see if it could provide any bot use. But it was certainly ahead of the Saturday boy and his well thumbed guide.

 

One week on these charts. Interesting profile on the dogs, start flat, end flat, with sharp rise Friday/Saturday. All figures are in line with previous period which is good.

170312

Aus170312

Another milestone was passed with these results, I became eligible to pay premium charge as my lifetime percentage dropped just below 20 to 19.92%. I’d already used some of my allowance which I think was linked to data charges that are no longer used. So this week saw £1.98 taken off my allowance; at that rate it’ll be 9 years before I actually pay anything. Unfortunately, if my total charges percent continues to fall, the weekly PC will rise. A drop to 19.72% would have seen a PC of £5.50. This is the price of (small) success. On a positive note this does put me in a bracket with 0.5% of customers which, if Wikipedia can be believed, is either 20,000 or 5,500 people. What joy.