Forums » Suggestions
Please provide some more LUA abstractions
Abstract plugin writers from the more difficult parts of common communications over the web. Simplifying things like:
Talking to discord API
Talking to remote databases directly
Talking to various web APIs for anything from news, RSS feeds etc..
We already have the ability to accomplish this with the tools we have. However, the most difficult parts are a turn off for hobbyists in their spare time. Any of these abstractions would be nice and having them properly set in the context of the game's loop would be ideal (this issue adds more complexity in VO-LUA).
Application abstraction layers that are useful but missing
1. HTTP/1.1 protocol
There was a half-hearted library made by Andy Sloan.. though I think it used HTTP 1.0. This protocol is fundamental to most APIs on the web today.
2. HTTPS
same as above but secure
3. Websockets
4. SSL
Secure socket is obvious as well
5. TSL
secure data transport
6. TDS protocol (Tabular Data Stream).
This protocol is used to communicate with databases like mySQL and others. It is also complicated and a nightmare to implement on your own with minimal tools. It would be nice to have this layer available, but it may be stopping short just to provide this layer and not a complete solution.
7. POP3
This is not really a thing but it is worth mentioning since it does allow for a game client to receive messages that were left when they were online via email protocols
8. RSS/ATOM and XML protocols
Up to date news feeds directly in the game client
9. any of the many chat protocols: WebRTC, IRC, XMPP, SIP/SIMPLE, AMQP, RCS
This is a lot of "stuff" to ask for, but for brevity and to avoid spamming the forum I'm putting the common stuff here and maybe will think to mention other things later.
I have been asked "what are you trying to accomplish?" when mentioning this type of stuff so i will try to answer that.
My goal is to use the plugin system to first resolve some issues that are not practical in the main game code for many reasons. Mobile platforms and their stores have a lot of rules about storage on the device etc.. However we have TCP and UDP protocols already available.
We can therefore begin to write plugins that interact with remote storage directly, and these issues about having unique files for storing plugin data will begin to vanish. This includes direct communication with SQL servers. As you may be aware, you can actually get a FREE database online and that could be used without cost to store data from your plugin.. but first you have to talk to it. Talking to it is the hard part and it starts with TDS protocol as mentioned above and then the next and necessary abstraction from that which we call lua SQL or whatever. The code is available that is proven and tested on LUAjit but I'm not sure about practicality in game context for the way those libraries are written.. async and concurrency and threading are kind of a "thing" in VO programming.
Another issue that these begin to solve is communicating with online APIs that are common and we can connect our wonderful game safely to the outside world. These include interfaces to all the most popular social sites that are the current fad and contain millions or billions of users.. like:
Facebook 2.8 billion Monthly Active Users (MAU)
Youtube 2.3 billion MAU
WhatsApp 2 billion MAU
Messenger 1.3 billion MAU
WeChat 1 billion MAU
Instagram 1 billion MAU
TikTok 1 billion MAU
QQ 807 million MAU
Qzone 532 million MAU
Sina Weibo 520 million MAU
Pinterest 459 million MAU
Tumblr 437 million MAU
Reddit 430 million MAU
Twitter 225 million MAU
LinkedIn 310 million MAU
Douban 300 million MAU
Baidu Tieba 300 million MAU
Snapchat 287 million MAU
Viber 260 million MAU
Discord 250 million MAU
Vimeo 240 million MAU
Telegram 200 million MAU
Line 165 million MAU
Twitch 140 million MAU
Medium 60 million MAU
This has the potential to bring attention to the game of course, but it is also quite enriching to your users to have some of these possibly. If you can give us some of the lower level breadcrumbs, we can begin to write plugins that get us to that list.. and what comes of it will no doubt be the ones that interest your users first.
I'll stop here for now.. since this is turning into a small book, and see who pops in to critique or give further input.
Talking to discord API
Talking to remote databases directly
Talking to various web APIs for anything from news, RSS feeds etc..
We already have the ability to accomplish this with the tools we have. However, the most difficult parts are a turn off for hobbyists in their spare time. Any of these abstractions would be nice and having them properly set in the context of the game's loop would be ideal (this issue adds more complexity in VO-LUA).
Application abstraction layers that are useful but missing
1. HTTP/1.1 protocol
There was a half-hearted library made by Andy Sloan.. though I think it used HTTP 1.0. This protocol is fundamental to most APIs on the web today.
2. HTTPS
same as above but secure
3. Websockets
4. SSL
Secure socket is obvious as well
5. TSL
secure data transport
6. TDS protocol (Tabular Data Stream).
This protocol is used to communicate with databases like mySQL and others. It is also complicated and a nightmare to implement on your own with minimal tools. It would be nice to have this layer available, but it may be stopping short just to provide this layer and not a complete solution.
7. POP3
This is not really a thing but it is worth mentioning since it does allow for a game client to receive messages that were left when they were online via email protocols
8. RSS/ATOM and XML protocols
Up to date news feeds directly in the game client
9. any of the many chat protocols: WebRTC, IRC, XMPP, SIP/SIMPLE, AMQP, RCS
This is a lot of "stuff" to ask for, but for brevity and to avoid spamming the forum I'm putting the common stuff here and maybe will think to mention other things later.
I have been asked "what are you trying to accomplish?" when mentioning this type of stuff so i will try to answer that.
My goal is to use the plugin system to first resolve some issues that are not practical in the main game code for many reasons. Mobile platforms and their stores have a lot of rules about storage on the device etc.. However we have TCP and UDP protocols already available.
We can therefore begin to write plugins that interact with remote storage directly, and these issues about having unique files for storing plugin data will begin to vanish. This includes direct communication with SQL servers. As you may be aware, you can actually get a FREE database online and that could be used without cost to store data from your plugin.. but first you have to talk to it. Talking to it is the hard part and it starts with TDS protocol as mentioned above and then the next and necessary abstraction from that which we call lua SQL or whatever. The code is available that is proven and tested on LUAjit but I'm not sure about practicality in game context for the way those libraries are written.. async and concurrency and threading are kind of a "thing" in VO programming.
Another issue that these begin to solve is communicating with online APIs that are common and we can connect our wonderful game safely to the outside world. These include interfaces to all the most popular social sites that are the current fad and contain millions or billions of users.. like:
Facebook 2.8 billion Monthly Active Users (MAU)
Youtube 2.3 billion MAU
WhatsApp 2 billion MAU
Messenger 1.3 billion MAU
WeChat 1 billion MAU
Instagram 1 billion MAU
TikTok 1 billion MAU
QQ 807 million MAU
Qzone 532 million MAU
Sina Weibo 520 million MAU
Pinterest 459 million MAU
Tumblr 437 million MAU
Reddit 430 million MAU
Twitter 225 million MAU
LinkedIn 310 million MAU
Douban 300 million MAU
Baidu Tieba 300 million MAU
Snapchat 287 million MAU
Viber 260 million MAU
Discord 250 million MAU
Vimeo 240 million MAU
Telegram 200 million MAU
Line 165 million MAU
Twitch 140 million MAU
Medium 60 million MAU
This has the potential to bring attention to the game of course, but it is also quite enriching to your users to have some of these possibly. If you can give us some of the lower level breadcrumbs, we can begin to write plugins that get us to that list.. and what comes of it will no doubt be the ones that interest your users first.
I'll stop here for now.. since this is turning into a small book, and see who pops in to critique or give further input.
We can probably begin to zero in on a lot of this by just hitting the big ones first. Perhaps a good place to start is:
HTTP/1.1 with HTTPS available
XMPP covers most of the larger chat networks these days
XML - I remember seeing a function for this but have not tested if it encodes/decodes xml
This is like the "short list" that will have the largest impact on possibilities...
With a properly complete HTTP/HTTPS layer we gain easier access to much as well.. including your own RSS stream :)
HTTP/1.1 with HTTPS available
XMPP covers most of the larger chat networks these days
XML - I remember seeing a function for this but have not tested if it encodes/decodes xml
This is like the "short list" that will have the largest impact on possibilities...
With a properly complete HTTP/HTTPS layer we gain easier access to much as well.. including your own RSS stream :)
Regarding XML specifically, i'm not sure if there are MORE functions, but this was the first hit in the documentation in Racecar2
Yeah that's the one I mentioned.. I have not tested it myself but I did see it there.
Haxmeister.. yesterday, I told you: It is best to express your needs the way draugath does. Express the issue, the goals, and an example case, as reasonably succinctly as possible.
Did you read draugath's thread that I linked to? Do you consider this an example of being specific & succinct?
This appears to be another kitchen-sink request, very similar to what you did yesterday, but slightly altered.
Did you read draugath's thread that I linked to? Do you consider this an example of being specific & succinct?
This appears to be another kitchen-sink request, very similar to what you did yesterday, but slightly altered.
you know incarnate.. I'm sorry I ever asked you to do a damn thing.. just remove it if it's so bad.
you know incarnate.. I'm sorry I ever asked you to do a damn thing.. just remove it if it's so bad.
Interesting. I am also sorry you asked, it has needlessly burned up a bunch of my time.
I've been asking you to do the same simple thing for a couple of days now, and you either outright refuse, or you "apologize" and then go do the same thing again.
You seem to be only interested in what you want, and not in what I want, which is fundamentally incompatible with posting here.
The Vendetta Online "Plugin" API was created to allow players to be able to tinker with UI and UX ideas, prototyping concepts and proving their value, before they were eventually raised to the developers, who would then professionally re-implement similar concepts from scratch, and integrate them with the main game.
I wrote about this as recently as yesterday, in Greenwall's thread about a unified and managed "plugin-app-store":
I appreciate the premise of Greenwall's Suggestion, plugins can create some advantage. But, historically, the idea has been to then take those advantages and push them back into the game client, which we've done a number of times.
Or, here's another quote from 2016:
..that's really what the plugin stuff is supposed to be about.. enabling experimentation by the users which may then inform developer-created improvements to the main game.
Great examples of this usage are concepts like draugath's "DroidButtons" to improve touch UIs. Or, the classic "TargetLess", some features of which have now been integrated into the main VO interface.
The reason why we re-implement features, and never use actual Plugin code in the game source, is partially because we have access to more optimized and efficient solutions to problems, partially for security reasons (clean-sheet implementation); but primarily because:
Most plugin code is pretty bad, as most plugin authors are inexperienced developers.
There are exceptions to this, for instance, people like draugath are experienced, but they are by-far the rarer cases in the overall community.
And, of course, we also recognize the many limitations and flaws and stability-challenges of the plugin API itself.. but let's be candid, that's not what we're talking about here.
I don't say this to denigrate inexperienced developers: we actually like fostering new developers in their creativity and interest, and some people who started out writing VO plugins have gone on to become robust programmers later in life (Firemage, for example).
The Plugin API was never intended to support the player-development of major game features. Like, for example, a Discord or Facebook interface. There are a couple of reasons for that:
- The first you can guess at from above.. a lot of plugins are badly written, slow, and otherwise problematic. Having other players be dependent on such community-supported work, for major features, would be undesirable. Similarly, if we (the developers) end up having to fix other people's code, that is not faster, or easier, than writing it ourselves. It is generally worse for our time, not a savings.
- The second comes from "relative cost / benefit tradeoff". While we made the plugin API to open up new possibilities with UI/UX design and the like, we did NOT open it up for:
1) Alt hunting databases where player-organizations track charIDs, and then verbally mock and abuse users who get their character names changed but fail to escape the ire of the respective individual(s) or guild(s) who are tracking them.
2) Proprietary and exclusionary accumulated information the yields a substantial gameplay advantage to only a limited subset, such as spybot networks, specialized economic, universe or activity tracking, etc.
3) Chat systems that subvert the administrative oversight of the game, such as our historical problems with IRC-gateways that would be used to avoid mutes and bans and so on. Having lots of Discord gateways does not improve on that.
I could go on and on, but I'll stop there for the sake of limited time.. the point here is:
It has been suggested that more-advanced community plugin development will provide some kind of "panacea" solution to having a limited number of Guild Software employees. That more comprehensive connectivity, data aware and feature-rich plugin API will result in an amazing assortment of new community-created features that players want, which will apparently be universally-available, well-written, stable, secure and well-maintained.
But while some of this might prove true in limited examples.. much of our history, to date, does not really support this notion.
For example, how much time do you think I've spent, as lead developer, dealing with issues, drama and problems that stemmed from the three categories I listed above? A hundred hours? A thousand? More?
How about this week? I mean, that is one of the central reasons behind Greenwall's repository suggestion, to make all plugins universally available to all, to eliminate plugins that give advantages to specific player subsets
I think it is entirely likely that the plugin API has done more damage to the productivity and development-progress of Vendetta Online, than it has ever been a net-benefit (even counting great content like DroidButtons). One hopes the value-add in retention offsets things, but I don't have a good sense of it in my head.. while I do know I've spent an astronomical amount of time dealing with plugin-related dramatic BS.. much of which is "behind the scenes" in tickets, log analysis, and other areas.
We generally approach development optimistically, and this has always included plugins: we open stuff up, let people play around, and see what happens. We actually aren't naive about it, but we're hopeful, and I don't think that's necessarily wrong. I would prefer that to being a defensive thinker, who sees so many demons everywhere that they can never try innovating.
Optimism is why the TCP support is available in the API. It is technically possible to do all kinds of stuff with the TCP access. But it isn't super-easy, and we obviously knew that.
The plugin API is supposed to be a limited sandbox. I said that already yesterday, when I wrote:
Understand, the fact that the VO Lua API availability is a limited subset is completely intentional. It is called a "sandbox" for a reason.
Some people will take this post super pessimistically and assume the worst, so clearly read this:
- None of this means I'm getting rid of plugins. None of the-above is "news" to me, I'm intimately familiar with these issues, but I felt like "The Obvious" needed to be restated, as it seemed like it had become forgotten or overlooked. I don't have that luxury, as I internally deal with the "problems" of plugins on a frequent basis.
- This also doesn't mean I'm against adding, say, "HTTPS" to the plugin system. But it means we are pretty goddamn circumspect about what you intend to do with more advanced features, and you should expect that. So, when you propose something, you had better have a specific goal that is not "Amazing Stuff!@!#".
- Don't expect me to giggle with excitement about player-created plugins that tie into all kinds of online networks, databases, social media points and other content. Thus far, what results we've seen from that have been, let's say, "a very mixed blessing".
- The plugin system (and API) will also be changing. I already mentioned bits of that: some of it is "under the hood" (performance), others related to client stability, but we're certainly aware of security considerations and other challenges (like increasingly challenging mobile installation, etc).
The fact that the plugin system, and API, are changing.. is why I've been asking for desired changes and the like. But, if I tell you I want specific and succinct cases, examples and considerations, limited to single threads, you should listen to that, or just.. please stay off the forum entirely.
Interesting. I am also sorry you asked, it has needlessly burned up a bunch of my time.
I've been asking you to do the same simple thing for a couple of days now, and you either outright refuse, or you "apologize" and then go do the same thing again.
You seem to be only interested in what you want, and not in what I want, which is fundamentally incompatible with posting here.
The Vendetta Online "Plugin" API was created to allow players to be able to tinker with UI and UX ideas, prototyping concepts and proving their value, before they were eventually raised to the developers, who would then professionally re-implement similar concepts from scratch, and integrate them with the main game.
I wrote about this as recently as yesterday, in Greenwall's thread about a unified and managed "plugin-app-store":
I appreciate the premise of Greenwall's Suggestion, plugins can create some advantage. But, historically, the idea has been to then take those advantages and push them back into the game client, which we've done a number of times.
Or, here's another quote from 2016:
..that's really what the plugin stuff is supposed to be about.. enabling experimentation by the users which may then inform developer-created improvements to the main game.
Great examples of this usage are concepts like draugath's "DroidButtons" to improve touch UIs. Or, the classic "TargetLess", some features of which have now been integrated into the main VO interface.
The reason why we re-implement features, and never use actual Plugin code in the game source, is partially because we have access to more optimized and efficient solutions to problems, partially for security reasons (clean-sheet implementation); but primarily because:
Most plugin code is pretty bad, as most plugin authors are inexperienced developers.
There are exceptions to this, for instance, people like draugath are experienced, but they are by-far the rarer cases in the overall community.
And, of course, we also recognize the many limitations and flaws and stability-challenges of the plugin API itself.. but let's be candid, that's not what we're talking about here.
I don't say this to denigrate inexperienced developers: we actually like fostering new developers in their creativity and interest, and some people who started out writing VO plugins have gone on to become robust programmers later in life (Firemage, for example).
The Plugin API was never intended to support the player-development of major game features. Like, for example, a Discord or Facebook interface. There are a couple of reasons for that:
- The first you can guess at from above.. a lot of plugins are badly written, slow, and otherwise problematic. Having other players be dependent on such community-supported work, for major features, would be undesirable. Similarly, if we (the developers) end up having to fix other people's code, that is not faster, or easier, than writing it ourselves. It is generally worse for our time, not a savings.
- The second comes from "relative cost / benefit tradeoff". While we made the plugin API to open up new possibilities with UI/UX design and the like, we did NOT open it up for:
1) Alt hunting databases where player-organizations track charIDs, and then verbally mock and abuse users who get their character names changed but fail to escape the ire of the respective individual(s) or guild(s) who are tracking them.
2) Proprietary and exclusionary accumulated information the yields a substantial gameplay advantage to only a limited subset, such as spybot networks, specialized economic, universe or activity tracking, etc.
3) Chat systems that subvert the administrative oversight of the game, such as our historical problems with IRC-gateways that would be used to avoid mutes and bans and so on. Having lots of Discord gateways does not improve on that.
I could go on and on, but I'll stop there for the sake of limited time.. the point here is:
It has been suggested that more-advanced community plugin development will provide some kind of "panacea" solution to having a limited number of Guild Software employees. That more comprehensive connectivity, data aware and feature-rich plugin API will result in an amazing assortment of new community-created features that players want, which will apparently be universally-available, well-written, stable, secure and well-maintained.
But while some of this might prove true in limited examples.. much of our history, to date, does not really support this notion.
For example, how much time do you think I've spent, as lead developer, dealing with issues, drama and problems that stemmed from the three categories I listed above? A hundred hours? A thousand? More?
How about this week? I mean, that is one of the central reasons behind Greenwall's repository suggestion, to make all plugins universally available to all, to eliminate plugins that give advantages to specific player subsets
I think it is entirely likely that the plugin API has done more damage to the productivity and development-progress of Vendetta Online, than it has ever been a net-benefit (even counting great content like DroidButtons). One hopes the value-add in retention offsets things, but I don't have a good sense of it in my head.. while I do know I've spent an astronomical amount of time dealing with plugin-related dramatic BS.. much of which is "behind the scenes" in tickets, log analysis, and other areas.
We generally approach development optimistically, and this has always included plugins: we open stuff up, let people play around, and see what happens. We actually aren't naive about it, but we're hopeful, and I don't think that's necessarily wrong. I would prefer that to being a defensive thinker, who sees so many demons everywhere that they can never try innovating.
Optimism is why the TCP support is available in the API. It is technically possible to do all kinds of stuff with the TCP access. But it isn't super-easy, and we obviously knew that.
The plugin API is supposed to be a limited sandbox. I said that already yesterday, when I wrote:
Understand, the fact that the VO Lua API availability is a limited subset is completely intentional. It is called a "sandbox" for a reason.
Some people will take this post super pessimistically and assume the worst, so clearly read this:
- None of this means I'm getting rid of plugins. None of the-above is "news" to me, I'm intimately familiar with these issues, but I felt like "The Obvious" needed to be restated, as it seemed like it had become forgotten or overlooked. I don't have that luxury, as I internally deal with the "problems" of plugins on a frequent basis.
- This also doesn't mean I'm against adding, say, "HTTPS" to the plugin system. But it means we are pretty goddamn circumspect about what you intend to do with more advanced features, and you should expect that. So, when you propose something, you had better have a specific goal that is not "Amazing Stuff!@!#".
- Don't expect me to giggle with excitement about player-created plugins that tie into all kinds of online networks, databases, social media points and other content. Thus far, what results we've seen from that have been, let's say, "a very mixed blessing".
- The plugin system (and API) will also be changing. I already mentioned bits of that: some of it is "under the hood" (performance), others related to client stability, but we're certainly aware of security considerations and other challenges (like increasingly challenging mobile installation, etc).
The fact that the plugin system, and API, are changing.. is why I've been asking for desired changes and the like. But, if I tell you I want specific and succinct cases, examples and considerations, limited to single threads, you should listen to that, or just.. please stay off the forum entirely.
wait, I don't understand your vision. Is there anything at all that is a potential path for UX improvement that would use the TCP connectivity you have provided?
I feel Incarnate has more than answered this topic despite it getting out of hand and not fitting with how we want the suggestions subforum to work. Since it seems that - regardless of reason or circumstance - there's going to be issues, I think it best this specific thread be locked.
wait, I don't understand your vision. Is there anything at all that is a potential path for UX improvement that would use the TCP connectivity you have provided?
Like Luxen says, I've about beaten this to death. I've said, over and over, to submit a specific goal and concept of what you want to do, and how/why, what the needs are; then we will evaluate whether we will support it, and how.
We do NOT want a general notion of I want to use SQL databases and "TSL"[sic], and then a bunch of accompanying random MAU numbers for social media outlets. I am aware of the existence of social media.
You also don't have to put so much emphasis on the "vision" word, it's just a concept that means that this is my game design. The takeaway there is that if I say explicitly "NO", don't waste everyone's time and energy arguing with me.
So, for instance, if I say "We're going to be using Lua 5.1 for all time", don't go on some random tear about how I'm apparently too old and dumb to understand "modern programming" and the value of updated features. That just makes you look like an idiot, and it pointlessly burns up energy and time.
The oldest rule of Suggestions is: I welcome feedback, BUT if I explicitly say "this is how shit will be, period": while clarification questions are reasonable, don't ****ing argue with me. Feel free to go play some other game, like World of Warcraft, and tell Blizzard how dumb they are for.. also still using "Lua 5.1".
On that note, I'm out. Thanks Luxen.
(PS, the language is named "Lua", it's Portuguese for Moon. It is not an acronym, it is not "LUA". But "JIT" on the other hand, actually is an acronym, for Just In Time).
Like Luxen says, I've about beaten this to death. I've said, over and over, to submit a specific goal and concept of what you want to do, and how/why, what the needs are; then we will evaluate whether we will support it, and how.
We do NOT want a general notion of I want to use SQL databases and "TSL"[sic], and then a bunch of accompanying random MAU numbers for social media outlets. I am aware of the existence of social media.
You also don't have to put so much emphasis on the "vision" word, it's just a concept that means that this is my game design. The takeaway there is that if I say explicitly "NO", don't waste everyone's time and energy arguing with me.
So, for instance, if I say "We're going to be using Lua 5.1 for all time", don't go on some random tear about how I'm apparently too old and dumb to understand "modern programming" and the value of updated features. That just makes you look like an idiot, and it pointlessly burns up energy and time.
The oldest rule of Suggestions is: I welcome feedback, BUT if I explicitly say "this is how shit will be, period": while clarification questions are reasonable, don't ****ing argue with me. Feel free to go play some other game, like World of Warcraft, and tell Blizzard how dumb they are for.. also still using "Lua 5.1".
On that note, I'm out. Thanks Luxen.
(PS, the language is named "Lua", it's Portuguese for Moon. It is not an acronym, it is not "LUA". But "JIT" on the other hand, actually is an acronym, for Just In Time).