SO, the SCADA controller is PNNL’s product VoltTron, which has an extremely naive SQL schema for MySQL and SQLite. (Most users have very low demands for their historians, and they can upload their data in real time to something better, so no big deal.)
I installed MonetDB and used the same schema, and it choked. No surprise.
I put in a more sensible schema, and on queries beyond a certain length, it meats MySQL hands down.
Seeing as I’m up this late only because I’m dealing with mitigating Kafka queue lag due to our big data team performing excessive complex queries on our OpenTSDB, this post couldn’t come across my screen at a better time, thanks! I’ll be interested in stress testing this soon. Has anyone here used this in a high-scale high-availability production environment yet?
Probably not as this seems to be in development still. You can read at the end of the document: “A clustered version is in private
beta with select customers.” Doesn’t sound like production-ready.
I would also like to have read something about cluster resizing and resilience to node failure, which are usually giant pain points in sharded data systems.
Either way this seems interesting and I’ll be looking for excuses to try it out.
I’m benchmarking an on-host historian for a SCADA controller.
I’ll be comparing stock MySQL to MonetDB to this one this week
Please blog and share it here, if you can. Would be a super interesting read :)
SO, the SCADA controller is PNNL’s product VoltTron, which has an extremely naive SQL schema for MySQL and SQLite. (Most users have very low demands for their historians, and they can upload their data in real time to something better, so no big deal.)
I installed MonetDB and used the same schema, and it choked. No surprise.
I put in a more sensible schema, and on queries beyond a certain length, it meats MySQL hands down.
Seeing as I’m up this late only because I’m dealing with mitigating Kafka queue lag due to our big data team performing excessive complex queries on our OpenTSDB, this post couldn’t come across my screen at a better time, thanks! I’ll be interested in stress testing this soon. Has anyone here used this in a high-scale high-availability production environment yet?
Probably not as this seems to be in development still. You can read at the end of the document: “A clustered version is in private beta with select customers.” Doesn’t sound like production-ready. I would also like to have read something about cluster resizing and resilience to node failure, which are usually giant pain points in sharded data systems. Either way this seems interesting and I’ll be looking for excuses to try it out.
Yep, Hadoop node failure and replication lag is what caused our problem last night.