Author Topic: MF Pro not responding  (Read 5283 times)

Tags:
  • All members
  • Posts: 11
  • Gender: Male
MF Pro not responding
« on: 04 Jan 2018, 18:08 »
Still new to MF Pro but I have plans to run a month's worth of Greyhound data through Time Machine. It would seem the data is too much as it just freezes - in Task Manager I get "Not Responding".

I have a powerful PC with 16GB RAM, an SSD set aside to hold the database. I don't know why a lot of data would be a problem.

I can break it down into weeks or days but it's going to lead to a lot of time and work on my part collating the data when I was hoping, you know, a computer would do it for me.

Any tips anyone ?


  • All members
  • Posts: 11
  • Gender: Male
Re: MF Pro not responding
« Reply #1 on: 06 Jan 2018, 20:39 »
Since I seem to be answering my own queries...

I found MF Pro initially because I was looking for data - when I found data I could purchase that needed an interface, i.e. Time Machine, to analyse I followed up.

It seems that I'm trying to achieve something that Time machine isn't well suited to doing - analysing a month's data in one go.

If anyone else encounters the same problem my advice is it runs ok loading 1 day at a time.

I tried 5 days in one go but it still hangs indefinitely just loading up the markets on start up.

  • All members
  • Posts: 53
Re: MF Pro not responding
« Reply #2 on: 29 May 2018, 07:49 »
I would like to bump up this topic. 

I am having a big project in mind and I would like to test hundreds of football matches on a regular basis. I think the idea of time machine is absolutely amazing, however, I have an impression that it has been abandoned for quite a long time.

Let me ask this question - how many matches one needs to analyse to say with high confidence that the trigger brings profit? 1000? 10000? less? more? I assume it is a high number and now come all the problems

1) From my observation I can see that matches are probably loaded from a giant file (tm.db) to the memory - note that about 35 football matches take over 1GB of disk space. In comparison, the history files that one can download from the official betfair website take about 5-6GB for the whole month worth of data. Now, if we want to run let's say 500 matches consecutively (not at once, but respecting the actual time difference as matches are played on different days), it is physically impossible to allocate so much data in the memory. It is frustrating that one has to allocate memory for matches that are not triggered but just queued. Therefore, one has to keep loading small amounts of markets manually to test the trigger.

2) There is no market locator like functionality in time machine, so one has to go through all the purchased markets and click each market individually to load it to the program. Meanwhile, the memory is getting full so that the program is getting super slow or not responding, making the whole process an absolute hell. Assuming one can import markets from a text file, is it even possible to download a full list of the purchased markets and parse the markets I am interested in and save as a text file? Or do I have to type in manually every single match in a notepad?

Assuming the problem number 2 can be somehow solved, is it possible to write a trigger that would import matches from a file automatically and delete the ones that are completed?

I hope someone could explain to me how this thing work. I see a great potential here and I hope that my worries are all exaggerated and there is a simple way to make it a powerful and efficient tool.

Thanks in advance

  • Administrator
  • Posts: 8821
  • Gender: Female
*
Re: MF Pro not responding
« Reply #3 on: 21 Aug 2018, 19:58 »
It seems that I'm trying to achieve something that Time machine isn't well suited to doing - analysing a month's data in one go.

Hello and sorry for overlooking your topic!

Yes, you are right, Time Machine is not intended for analysing a month's worth of data in one go. Rather it is intended for backtesting your triggers on historical markets in a slightly faster way than you would do in real mode. Also it allows you to rewind back to start, tweak your settings and re-run the triggers to see if that made any difference, without having to wait for the next sports event to start (e.g. there can be 10 to 15 minutes between horse races on a typical day).

You would need a specialised software to run a specific data analysis. As far as I know, even BetFair is currently struggling with processing these huge amounts of data. They have recently had 2 weeks of server crushing, data base failures, etc. Each market can "weigh" up to 20 MB, so you will understand why MF Pro is freezing when you are loading hundreds of them in one go.
Always try your triggers in Test Mode before switching to real money!

Follow us on Twitter.

Join our WhatsApp chat!

Присоединяйтесь к официальному Telegram-каналу!

  • Administrator
  • Posts: 8821
  • Gender: Female
*
Re: MF Pro not responding
« Reply #4 on: 21 Aug 2018, 20:17 »
Let me ask this question - how many matches one needs to analyse to say with high confidence that the trigger brings profit? 1000? 10000? less? more?

I don't think one can say with 100% confidence how many markets are needed to test your betting strategy. It depends on so many factors, e.g. how frequently does your trigger bet, whether you have had many losing or winning streaks, how your bank has changed over the time, whether you take high risks, etc.

1) From my observation I can see that matches are probably loaded from a giant file (tm.db) to the memory - note that about 35 football matches take over 1GB of disk space. In comparison, the history files that one can download from the official betfair website take about 5-6GB for the whole month worth of data.

You are probably referring to the Historical Data service. If we are talking about their free data, then the frequency is 1 min, whereas in Time Machine the frequency is 0.5 sec. This evidently explains the difference in the file size. The number of betting strategies you can test at 1 min is very limited.

I completely agree with all the points you have listed. We are trying to find a solution, but can't see it at the moment (BetFair is struggling with this amount of data too, see my message above). On the one hand, customers want the ability to test bets based on price fluctuations or in-play markets where prices change dramatically with each second, so 1-minute frequency is not an option.
On the other hand, everyone want to have the flexibility of loading a month's data and running their strategies within an instant.
We still need to find a technical balance between the two (very valid) customer expectations.

Among the options I can see here is the ability to run a cloud testing service where customers would load their triggers to our powerful server, specify the markets they want to test them against and choose the notification method (i.e. by email, Telegram, logs, etc.). This would obviously limit their ability to place bets manually or control the flow of Time Machine, but at the same time this would speed up the process several times. We are considering this route, but we need to study the demand for such service, as it will be pricey and will require a big investment of time and effort. Do you think such service would make sense?
Always try your triggers in Test Mode before switching to real money!

Follow us on Twitter.

Join our WhatsApp chat!

Присоединяйтесь к официальному Telegram-каналу!

  • All members
  • Posts: 53
Re: MF Pro not responding
« Reply #5 on: 21 Aug 2018, 20:34 »
Dear Oxa,

Thank you for replying and clarifying things. You have poured a little bit of hope into my heart. I agree with all the technical issues that you are mentioning. It is surely a very challenging task and you guys are doing a great job. I have been a big fan for years.

I hope you do not mind if I share an idea for a much simpler solution. You mentioned that TM is not designed to analyse months worth of data. Nonetheless, in my opinion it is not far from being able to. The biggest problem is the lack of market locator-like feature that would limit how many markets are monitored and triggered at once, but also allow to import them automatically.

Briefly, what we need is the following function, here written for 5 markets:

1) Generate a vector of starting times for each market located in the tm.db file
2) Import five (only) markets to MF Pro (starting from the earliest)
3) Test the trigger for the 5 imported markets
4) When finished, delete and clear memory
5) Import the next 5 markets
....
continue until all the markets have been settled

How hard do you think it would be to implement it in TM? I think with the ability of TM to speed up the in-play triggering this method could have some potential.

What do you think?

  • All members
  • Posts: 53
Re: MF Pro not responding
« Reply #6 on: 03 Sep 2019, 11:41 »
Hej Oxa,

Have you thought about some potential solutions? Just curious since it has been over a year since I posted.

Thanks,

LiAlH4

 

Please note, BetFair is seems to be currently OFFLINE