- Wed Apr 12, 2017 3:37 pm
#64885
Hi forum,
I was wondering if anyone fancied helping me test a solution I've been writing, it's going to be open source at some point when I get round to setting up a GitHub account.
It's written in dotnet core and is therefore crossplatform, I'm running it at home on my Ubuntu server and developing it on a Windows laptop but when I release the source code you should be able to run it on OSX too.
It's inspired by Sparkfun's Phant but I had issues with Phant. I wanted to bring back the last 48 hours of data for a query but because of the way it's setup in csv files there is no indexing. It would send the data back that I wanted in a few seconds but because it was still scanning the rest of the data (2 months worth of data being logged every min from 5 different sensors) it took another 40 seconds before it closed the connection and wrote the last closing bracket for my Json request.
So I've decided to write my own solution and get round this issue by using Sqlite for the database, this means it can have an index on timestamp. Each app gets their own Sqlite db and you can download the entire database if you want to do more complex queries or just get a backup.
I'm going away for a few days and probably won't be able to answer any questions.
Please also note that this is for testing so please don't rely on this to keep your data indefinitely, based on feedback I may make a change that means existing db's won't work, or I may run out of space or Azure credit.
http://totalrecallapp.azurewebsites.net
Thanks for your time and input.
I was wondering if anyone fancied helping me test a solution I've been writing, it's going to be open source at some point when I get round to setting up a GitHub account.
It's written in dotnet core and is therefore crossplatform, I'm running it at home on my Ubuntu server and developing it on a Windows laptop but when I release the source code you should be able to run it on OSX too.
It's inspired by Sparkfun's Phant but I had issues with Phant. I wanted to bring back the last 48 hours of data for a query but because of the way it's setup in csv files there is no indexing. It would send the data back that I wanted in a few seconds but because it was still scanning the rest of the data (2 months worth of data being logged every min from 5 different sensors) it took another 40 seconds before it closed the connection and wrote the last closing bracket for my Json request.
So I've decided to write my own solution and get round this issue by using Sqlite for the database, this means it can have an index on timestamp. Each app gets their own Sqlite db and you can download the entire database if you want to do more complex queries or just get a backup.
I'm going away for a few days and probably won't be able to answer any questions.
Please also note that this is for testing so please don't rely on this to keep your data indefinitely, based on feedback I may make a change that means existing db's won't work, or I may run out of space or Azure credit.
http://totalrecallapp.azurewebsites.net
Thanks for your time and input.
