Forums » Community Projects
New DB Plugin Idea
This is based off the old, now-defunct vokb.. Basically works like this: instead of relying on a website, the plugin stores it's own data, shares/compares that data with other plugin users when they are online.
The plugin could work in this way: everyone using the plugin would join a specific channel, when the plugin grabs new data, it simply gives the data, programmatically, to the other plugin users, by relaying the data over the chat channel.
The coding required for this to work would be quite an undertaking, as the plugin would also need to have some safeguards in place, to prevent old defunct versions of the plugn from polluting the data, and would also need some way of determining that the data did indeed come from a plugin, and not from some malicious player.
Now, this is not something, I myself, am prepared to do.. just throwing the idea out there in case some other author was thinking "What can I write today?" :)
break19
The plugin could work in this way: everyone using the plugin would join a specific channel, when the plugin grabs new data, it simply gives the data, programmatically, to the other plugin users, by relaying the data over the chat channel.
The coding required for this to work would be quite an undertaking, as the plugin would also need to have some safeguards in place, to prevent old defunct versions of the plugn from polluting the data, and would also need some way of determining that the data did indeed come from a plugin, and not from some malicious player.
Now, this is not something, I myself, am prepared to do.. just throwing the idea out there in case some other author was thinking "What can I write today?" :)
break19
Some discussion about this was going on in the #vo-lua channel a few days ago, so something should be around "Soon™", I suspect...
This is fairly easy to do and I have released a plugin that does this very same thing about last year sometimes (groupstat).
vokb did not rely on a website, it relied on a server (otherwise keeping it alive would have been much simpler).
A decentralized solution like yours does sound reasonable at first, but there are a lot of problems involved which make such an attempt quite overcomplicated.
1. Not all plugin users are online at the same time. You have to take care of updating your own local database when docking as well as all the other user's databases.
2. It would simply spam the chat for the plugin users (last time I checked, exchanging data without seeing it on the chat screen is impossible - as long as you don't hack if_lua or whatever that interface code is called).
3. There are (still?) 120 stations in VO, each of which sells probably about 50 items. That's 6000 records. Last time I checked, local file access or persistent data storage is not possible (and saving that amount of data to the config file isn't really the way to go). This also applies to 2.
No matter what the backend is, preventing false data from being submitted is quite impossible as long as you don't use some kind of authentication or trusted users. PostgreSQL rollbacks could be very useful. I still think a centralized solution is the most efficient and easiest method which also results in less bugs.
I'm an advocate of webserver based, (Postgre-)SQL backed solutions, as it is also extremely easy to to exchange data using JSON client- and server side. Plus the backend, server or whatever you want to call it can easily be moved to another host as webspace and databases are available everywhere.
I don't know what exactly you want to write (or not), but I'm guessing it's just a database which saves the prices of all items sold at every station in the universe? For me, all it used to be was an answer to the question "what's the fastest way to buy item xy from my current location". And I think such a plugin could be written in less than 6 hours (including a good database backend).
A decentralized solution like yours does sound reasonable at first, but there are a lot of problems involved which make such an attempt quite overcomplicated.
1. Not all plugin users are online at the same time. You have to take care of updating your own local database when docking as well as all the other user's databases.
2. It would simply spam the chat for the plugin users (last time I checked, exchanging data without seeing it on the chat screen is impossible - as long as you don't hack if_lua or whatever that interface code is called).
3. There are (still?) 120 stations in VO, each of which sells probably about 50 items. That's 6000 records. Last time I checked, local file access or persistent data storage is not possible (and saving that amount of data to the config file isn't really the way to go). This also applies to 2.
No matter what the backend is, preventing false data from being submitted is quite impossible as long as you don't use some kind of authentication or trusted users. PostgreSQL rollbacks could be very useful. I still think a centralized solution is the most efficient and easiest method which also results in less bugs.
I'm an advocate of webserver based, (Postgre-)SQL backed solutions, as it is also extremely easy to to exchange data using JSON client- and server side. Plus the backend, server or whatever you want to call it can easily be moved to another host as webspace and databases are available everywhere.
I don't know what exactly you want to write (or not), but I'm guessing it's just a database which saves the prices of all items sold at every station in the universe? For me, all it used to be was an answer to the question "what's the fastest way to buy item xy from my current location". And I think such a plugin could be written in less than 6 hours (including a good database backend).
Hm. Weird. I had this exact idea for my own plugin, an asteroid ore tracker (RoidHunter).
The reasoning for NOT using a centralized database is a) it costs money for anything decently reliable (hosting a database on the author's local machine is... less than reliable, heh), and b) if the centralized database ever dies or stops getting paid for, then someone has to release a patch.
The downsides, as you've mentioned, are valid. Spamming to the chat is... definitely the least elegant way to do it. At the same time, if done correctly, it's also the most reliable.
In my scenario, I'll be creating a begin transmission message containing a version number as well as an end transmission message. The only way for a client to begin receiving and updating the client's local data is if the transmitting user is in the client's trusted list and the version number is the same as the client's.
This vastly reduces the potential for exploit. And though this doesn't stop the spam, that can be mitigated by giving the user a clean interface for preparing/starting/ending the transmission and that joins/unjoins the specified channel when finished.
The idea is that it's not a constant automatic thing, but instead initiated through a conscious effort by the users...
Once I'm finished with my own version of the idea here, hopefully I won't disgrace my theories above ;)
But it's seriously weird that two different people had essentially the same idea on nearly the same days :D
The reasoning for NOT using a centralized database is a) it costs money for anything decently reliable (hosting a database on the author's local machine is... less than reliable, heh), and b) if the centralized database ever dies or stops getting paid for, then someone has to release a patch.
The downsides, as you've mentioned, are valid. Spamming to the chat is... definitely the least elegant way to do it. At the same time, if done correctly, it's also the most reliable.
In my scenario, I'll be creating a begin transmission message containing a version number as well as an end transmission message. The only way for a client to begin receiving and updating the client's local data is if the transmitting user is in the client's trusted list and the version number is the same as the client's.
This vastly reduces the potential for exploit. And though this doesn't stop the spam, that can be mitigated by giving the user a clean interface for preparing/starting/ending the transmission and that joins/unjoins the specified channel when finished.
The idea is that it's not a constant automatic thing, but instead initiated through a conscious effort by the users...
Once I'm finished with my own version of the idea here, hopefully I won't disgrace my theories above ;)
But it's seriously weird that two different people had essentially the same idea on nearly the same days :D
The spam can be easily eliminated by modifying the chatreceiver function. And while firsm is thankfully wrong and there are indeed ways of storing persistent data client-side, a centralized server is still by far the best way to do this. There's a reason open-source projects use CVS and SVN despite their downfalls. If everyone is making their own modifications on their own machines with nothing coordinating them, version control goes to hell and chaos reigns.
Whats the status of this?
Person, by saving data client side, are you refering to saving it to the config file in some pickled or jsoned format? If it is, then I have already outlined the problems of that in my previous post. Even if you had direct file system access, it would be slow for 6000+ records, as long as you don't use something like sqlite or cdb (http://cr.yp.to/cdb.html).
If you're looking for a free way to save such data reliable, you may want to give the new Google App Engine (http://code.google.com/appengine/) a shot. I think using that in conjunction with JSON (for which a Lua and Python module already exists) should be doable in two days.
If you're looking for a free way to save such data reliable, you may want to give the new Google App Engine (http://code.google.com/appengine/) a shot. I think using that in conjunction with JSON (for which a Lua and Python module already exists) should be doable in two days.