Changes in the pipeline.

For announcing, describing and discussing code changes to BNT.

Post Reply
User avatar
TheMightyDude
Site Admin
Posts: 311
Joined: Thu Apr 17, 2014 09:15 UTC

Changes in the pipeline.

Post by TheMightyDude » Sat Apr 19, 2014 10:39 UTC

Hello All

Due to all the issues resulting from the Server Update it has made me look into the code and how it is currently working and how it should be working.

I think some stuff is seems ok like I like how Composer works with the auto-loading of files and the ease of adding new packages to the list, the only issue that I do have is the overhead some of the composer packages have, I mean we may only require a small package to replace a small chuck of our code so we off load that to a set package so that we don't have to worry about that and can be rest assured that it will be kept update, I have no issues with that, the issue that I have is the packages that that package requires, what may of started off as say 2KB of PHP code now becomes 50KB or maybe more.

It was the same with the database parts of the code, we use to originally only support MySQL so we had hardcoded MySQL PHP code in the files which was fine due to we was only supporting MySQL.
But when we wanted to support other databases like MsSQL (Microsoft SQL), Postgress, Oracle and others we had to use ADODB which was great at first and was for a while until it stopped getting updated, bugs started to appear speed was reduced due to bugs and no matter what we reported to the author it was ignored, this went on for many years until we saw that ADODB was added to the Composer's packages list and did have some small changes, but we still had some issues, so we had to hack parts of the ADODB package and put it into our own local package which resolved that issue.

All the issues that we have in BNT I cannot pass the blame on all the authors of the packages that we are using, some are partly to blame, but most of the issues we have are due to our code, I am not happy about it but I can admit our code sucks and really needs to change.
We (thekabal and myself) mostly thekabal did loads of changes to the code to resolve this, I even at one point did run a version which got updated on a daily basis but had to close that game due to the amount of changes which broke a fair amount of the code until that part was completed.

To be honest I really did want to recode the entire game from scratch a while ago, and was to use completely new clean easy to read commented code.
This code was to be coded in OOP style i.e. all classed up, so anything to-do with the ship will be stored in the Ships class and all Combat related stuff will be in the Combat Class and so on.
And each of these classes if done right will load up the required sub modules (code for the classes) when they need to be used resulting in smaller files to load.

We may still use Smarty Templates for the HTML stuff in the browser, but I have plans to remove that in the future in favour to using the Unity 3D Engine.
The Unity 3D Engine is free which ticks my box ;) and can run from the browser, Windows, Windows Phone, Mac, Linux, iPhone, iPad, Android Phones, Android devices, and also [Xbox, PS3, Nintendo Wii (if have dev licence)]
The cool thing with Unity is that we could have 3D stuff added, however we are still limited to how the game works, i.e. we request a page and we get send back the result, but this isn't seen by the user playing.

So all this got me thinking about doing a complete recode the game from scratch, that game play "shouldn't" change at all, just how it works in the browser, the only issue is that the user will need to install the Unity Webplayer Plugin when it pops up.

Now me being realistic...
Its only me now, so this may be a slow process and to even make things worse as some of you are aware I have been rather ill the last few years, so there maybe days where I don't feel well enough to do any coding and stuff.

[Account, Character, Ship Table separation]
I plan to have the Account, Character and Ship information all separated to their own table in the database, this is so that when the game is reset the Account information is still available and the user only needs to login and will be told that they have no Character in the game where they then create one and then start to play the game, This will reduce the amount of emails being sent from this server.
Also having the Character and Ship information separated will allow for Multi-Ship Support maybe later on.

[Password Recovery]
This will be changed, due to the nature of the code change having the password emailed to you will be impossible, so changes will need to be done to allow the user to be able to request for a password change.
This will result in an email being sent to the users email address supplied with a link which will take the user to a page and allow them to enter the new password twice (2nd for validation) and then a 2nd email will be sent to the users email address with the new password displayed.

There is loads more that I could say, but this post is getting large as it is.
TheMightyDude::Blacknova Development.
Development Blog YouTube Dev Channel Twitter Twitch

thekabal
Posts: 100
Joined: Sat Apr 19, 2014 22:32 UTC

Re: Changes in the pipeline.

Post by thekabal » Sat Apr 19, 2014 23:10 UTC

TheMightyDude wrote: I think some stuff is seems ok like I like how Composer works with the auto-loading of files and the ease of adding new packages to the list, the only issue that I do have is the overhead some of the composer packages have, I mean we may only require a small package to replace a small chuck of our code so we off load that to a set package so that we don't have to worry about that and can be rest assured that it will be kept update, I have no issues with that, the issue that I have is the packages that that package requires, what may of started off as say 2KB of PHP code now becomes 50KB or maybe more.
None of the packages we are using now are using dependencies, so that isn't the actual issue.

The actual issue is that two of the packages we are using (Smarty, and HTMLPurifier) have some unfortunate packaging in Packagist/Composer. Short version: The particular "distribution" of those packages we are using include a HUGE amount more stuff than they need to. By which I mean, our smarty vendor dir includes documentation & development, which are an extra *14 MEGS* of code than we actually "need". Its similar for HTMLPurifier.

Composer was only added to BNT about a year ago, and we haven't done a release since its addition. We shouldn't actually include the stuff under vendor/ in our source control anyway. The way composer is supposed to work, is that the developers run composer update, and then when people install a release of the game, they do too. Since that is very different to how we've always done things, I took shortcuts that I shouldn't have. I'll be working on improving that overall process in the Advanced branch.

But for Original, what I'd suggest is putting together new "packages" (in Packagist / Composer) that are actually the stripped down versions of those two libraries. Doing so would save something like 25 megs out of the vendor directory. None of the other items use dependencies or have full installs, so it shouldn't be an issue.

Another alternative is to simply switch the parts of the composer.json for those two libraries to use autoload/classmap instead of composer auto-loading, and redo the directories yourself (using the bare minimum install of both). That is just one step short of making a package for Composer, though, so, no big difference.
TheMightyDude wrote: All the issues that we have in BNT I cannot pass the blame on all the authors of the packages that we are using, some are partly to blame, but most of the issues we have are due to our code, I am not happy about it but I can admit our code sucks and really needs to change.
Agreed! I think it is fair to say that almost all of the contributors/developers recognize that the code has serious shortcomings! But SOME of the code is really getting very, very good.
TheMightyDude wrote: To be honest I really did want to recode the entire game from scratch a while ago, and was to use completely new clean easy to read commented code.
This code was to be coded in OOP style i.e. all classed up, so anything to-do with the ship will be stored in the Ships class and all Combat related stuff will be in the Combat Class and so on.
And each of these classes if done right will load up the required sub modules (code for the classes) when they need to be used resulting in smaller files to load.
I wish you luck in that process!
TheMightyDude wrote: Its only me now, so this may be a slow process and to even make things worse as some of you are aware I have been rather ill the last few years, so there maybe days where I don't feel well enough to do any coding and stuff.
:( Get well soon, I hope.

Techbot
Posts: 1
Joined: Thu Apr 24, 2014 10:34 UTC

Re: Changes in the pipeline.

Post by Techbot » Thu Apr 24, 2014 10:48 UTC

Hello THeMightyDude and thekabal,

I am very new to blacknova. I am interested in the topic of development rpg/trading games. I am currently in the process of a prototype that I intend to rewrite. My engine lacks the depth that blacknova has (essentially mine is largely style over substance at present). I also lack the long term experience of running a game .

However it is possible I might have some new ideas to bring to the table. The most unique thing I'm looking at is modularity. I can turn widgets on and off from the backend or move them to different module containers. My goal is for the gamemaster to decide on the theme eg space or fantasy and to decide what elements of the game might not be used eg magic/mana .

I do not want to show disrespect by discussing my engine on your forum, but if you guys would like to discuss various aspects .

Maybe this belongs in a subforum on general multiplayer developement discussions.

My musings on the matter can be found here: http://emc23.com/tutorials/34-tutorials ... me-scripts

User avatar
TheMightyDude
Site Admin
Posts: 311
Joined: Thu Apr 17, 2014 09:15 UTC

Re: Changes in the pipeline.

Post by TheMightyDude » Thu Apr 24, 2014 15:41 UTC

thekabal wrote:
TheMightyDude wrote: I think some stuff is seems ok like I like how Composer works with the auto-loading of files and the ease of adding new packages to the list, the only issue that I do have is the overhead some of the composer packages have, I mean we may only require a small package to replace a small chuck of our code so we off load that to a set package so that we don't have to worry about that and can be rest assured that it will be kept update, I have no issues with that, the issue that I have is the packages that that package requires, what may of started off as say 2KB of PHP code now becomes 50KB or maybe more.
None of the packages we are using now are using dependencies, so that isn't the actual issue.
No, I was refering to the database package (doctrine/dbal) that we was thinking about using a little while back, I also did a look through all the packages they have seeing what ones may be cool to use in the re-code, and a lot of them had dependencies and a few of them also required some dependencies.
Thats what I was refering too.
thekabal wrote:The actual issue is that two of the packages we are using (Smarty, and HTMLPurifier) have some unfortunate packaging in Packagist/Composer. Short version: The particular "distribution" of those packages we are using include a HUGE amount more stuff than they need to. By which I mean, our smarty vendor dir includes documentation & development, which are an extra *14 MEGS* of code than we actually "need". Its similar for HTMLPurifier.
Yep, wasn't a fan of that either, also the pathing to the actual folder with the package code in it was rather long as well.
Like for example /vendor/smarty/smarty/distribution/libs/ I know its a whole package and we are to just path to the relevent folder where the code is, but /smarty/smarty/ (extra folder) yeah not a fan of that.
thekabal wrote:Composer was only added to BNT about a year ago, and we haven't done a release since its addition. We shouldn't actually include the stuff under vendor/ in our source control anyway. The way composer is supposed to work, is that the developers run composer update, and then when people install a release of the game, they do too. Since that is very different to how we've always done things, I took shortcuts that I shouldn't have. I'll be working on improving that overall process in the Advanced branch.
See thats where the issue is as well, not all admins that want to run BNT have access to shell prompt, like some of the free providers don't give that access (cannot remember which ones) so those admins wouldn't be able to install the required packages.
thekabal wrote:But for Original, what I'd suggest is putting together new "packages" (in Packagist / Composer) that are actually the stripped down versions of those two libraries. Doing so would save something like 25 megs out of the vendor directory. None of the other items use dependencies or have full installs, so it shouldn't be an issue.

Another alternative is to simply switch the parts of the composer.json for those two libraries to use autoload/classmap instead of composer auto-loading, and redo the directories yourself (using the bare minimum install of both). That is just one step short of making a package for Composer, though, so, no big difference.
Yeah, I was thinking about that, it shouldn't be that much of an issue and if that package is updated we can either stay with the current re-ordered package or update the current package with the changes.
thekabal wrote:
TheMightyDude wrote: All the issues that we have in BNT I cannot pass the blame on all the authors of the packages that we are using, some are partly to blame, but most of the issues we have are due to our code, I am not happy about it but I can admit our code sucks and really needs to change.
Agreed! I think it is fair to say that almost all of the contributors/developers recognize that the code has serious shortcomings! But SOME of the code is really getting very, very good.
This is true.
thekabal wrote:
TheMightyDude wrote: To be honest I really did want to recode the entire game from scratch a while ago, and was to use completely new clean easy to read commented code.
This code was to be coded in OOP style i.e. all classed up, so anything to-do with the ship will be stored in the Ships class and all Combat related stuff will be in the Combat Class and so on.
And each of these classes if done right will load up the required sub modules (code for the classes) when they need to be used resulting in smaller files to load.
I wish you luck in that process!
Yeah, I am going to need it.
At least one good thing about a re-code is that you cannot break anything :P
thekabal wrote:
TheMightyDude wrote: Its only me now, so this may be a slow process and to even make things worse as some of you are aware I have been rather ill the last few years, so there maybe days where I don't feel well enough to do any coding and stuff.
:( Get well soon, I hope.
Thanks.
TheMightyDude::Blacknova Development.
Development Blog YouTube Dev Channel Twitter Twitch

thekabal
Posts: 100
Joined: Sat Apr 19, 2014 22:32 UTC

Re: Changes in the pipeline.

Post by thekabal » Sun Apr 27, 2014 01:13 UTC

TheMightyDude wrote: No, I was refering to the database package (doctrine/dbal) that we was thinking about using a little while back, I also did a look through all the packages they have seeing what ones may be cool to use in the re-code, and a lot of them had dependencies and a few of them also required some dependencies.
Thats what I was refering too.
OH, yeah, that is one of the reasons I didn't choose to go with DBAL. ;)
TheMightyDude wrote: Yep, wasn't a fan of that either, also the pathing to the actual folder with the package code in it was rather long as well.
Like for example /vendor/smarty/smarty/distribution/libs/ I know its a whole package and we are to just path to the relevent folder where the code is, but /smarty/smarty/ (extra folder) yeah not a fan of that.
Not a huge deal to me, but yeah, that is another issue that could be fixed by doing a custom distribution of the package.
TheMightyDude wrote: See thats where the issue is as well, not all admins that want to run BNT have access to shell prompt, like some of the free providers don't give that access (cannot remember which ones) so those admins wouldn't be able to install the required packages.
That part gets a little tricky. You "could" kick off composer from another script (say, a theoretical install.php), but in practice no one does because it is rather not ideal. And yeah, I agree, that is why I leaned towards using Composer to GET the dependencies (during development), but then not using it during deployment. That is rather different than how Composer is generally used, but still fits the intended usage.
TheMightyDude wrote: Yeah, I was thinking about that, it shouldn't be that much of an issue and if that package is updated we can either stay with the current re-ordered package or update the current package with the changes.
Right. Both Smarty and HTML purifier are rather stable and mature, so they don't update very often - maybe 2-3 times in a year, tops. So building a package in Packagist for each of our releases (at best 2-3 times a year) isn't overly harsh.

thekabal
Posts: 100
Joined: Sat Apr 19, 2014 22:32 UTC

Re: Changes in the pipeline.

Post by thekabal » Sun Apr 27, 2014 01:48 UTC

Techbot wrote:Hello THeMightyDude and thekabal,

I am very new to blacknova. I am interested in the topic of development rpg/trading games. I am currently in the process of a prototype that I intend to rewrite. My engine lacks the depth that blacknova has (essentially mine is largely style over substance at present). I also lack the long term experience of running a game .
Welcome! The long term experience of running the game brings certain interesting realizations, but so does developing for over a decade. We've seen trends come and go. Some good, some.. not so much. ;)

TMD can speak better to running a game for a long time. He has kept BNT alive virtually single-handedly for at least half a decade. One unfortunate aspect is lots of people that cheat, whether by using multiple accounts, or using scriptable interfaces as browsers, or other nefarious trickery. He has made some reasonable rules that help keep things reasonably managable, but he has also dealt with his fair share of difficulties.

We ended up dealing with licensing in our game code for just such reasons. A few forks decided to take our code, make improvements, not share them, AND deeply insult our code and history while doing it. We don't mind people standing on our shoulders, but very much prefer they not piss on us while they do it! :)

So our game code license changed to the AGPLv3, which (among other things) basically requires that if anyone runs a game of our code, ALL of their changes must be made available under the AGPLv3 also. We started as GPLv2, and wanted to be nice, but that didn't work out. Then we tried GPLv3, but again, people used the loophole to abuse us. Finally, we made legally clear what we meant all along, which is pretty much what every kid learns in kindergarten - Share and Share alike. If you want others to share (and we do!), then you have to also.

That license is very different from other web based games (like for example Legend of the Green Dragon). Many other games have licenses that are either more restrictive, or flat out incompatible with our license. Be careful as you proceed with multi-game development that you do not mix code from one license base to another. The licenses can be very tricky, and you'd do well to avoid legal trouble - another issue that has come up over the years here.
Techbot wrote:However it is possible I might have some new ideas to bring to the table. The most unique thing I'm looking at is modularity. I can turn widgets on and off from the backend or move them to different module containers. My goal is for the gamemaster to decide on the theme eg space or fantasy and to decide what elements of the game might not be used eg magic/mana .
New ideas are welcome.

We've headed in the general direction of modularity for a while now. TMD has been working on Plug-in code for over a year now I think, with varying implementations. I will admit readily that I am not a huge fan of the concept in general of hooks and plugins overall, as I find they add a tremendous amount of overhead (in coding, and other ways) for abstract "some day" functionality that is rarely used (and when it is, it doesn't use nearly all of the hooks that are added).

I generally think (especially in terms of BNT Advanced) that it is better to provide a well defined interface that developers can use to build additions to the game. Well defined interfaces are very different than "call general hook" sprinkled every 100 lines or so throughout a code base.

In addition, I've done a tremendous amount of work towards moving the code to be much, much less monolithic. The Composer autoloader we've been discussing is one big step in that direction, because it lets us strip a tremendous amount of includes out of files, and basically load "class" files (only if and) as they are used. Another step in that direction is cleaning up the individual classes to be single-purpose, with low cyclomatic complexity (logic branches like if/then/case, etc), and with clean inputs and outputs. Those make the components of the game easier to test, easier to split away from the rest of the game, and thus be truly modular.

We also went fairly far down the path of splitting the game output to use templates, which allows more of the visible part of the game to be adaptable.

But all of these are really focused on our core game, and not towards making it "chop and drop"-able into other genre's. There was a board-and-sword fork of BNT for a while (can't remember the name), where it was all re-themed nautically. That was kind of neat, just because it happened back before we did any of our major work on modularity, so it was probably a tremendous challenge for them to accomplish.

Over the years, many games have caught my eye, including several that you are running. Way back in the day, when I was working on The Kabal Invasion, one of the primary goals was "Nova Gaming System", which was going to be much like a "framework" is today. It would have allowed multiple games to use its code for things like login, hall of fame, scores, and common routines like templates, SQL CRUD routines, and things of that sort.

Or put another way, I wanted to have the guts of the games in an abstracted container ("NGS"), and then the games would just be a collection of game specific pieces like definitions of commodities, combat routines, things of that sort.

However, several things happened. The web based game movement pretty much died with the advent of Free to Play MMO's with graphical quality to rival WoW. That reduced a lot of the fan base and thus interest in the game scene. A few years later, PHP frameworks started becoming a force to be respected in their own right, reducing the need to develop one specific to game engines. And finally, I personally became overwhelmed just in trying to do a single game competing against our forks, so I stepped away to focus on my family.

Now is an interesting time for you to go down that path. I hope you find very different results on the way, and I'm curious to see where you end up. :)
Techbot wrote:I do not want to show disrespect by discussing my engine on your forum, but if you guys would like to discuss various aspects .
We've done that before on the forums, the world doesn't end. But please do open new threads with a new topic, at least.
Techbot wrote:My musings on the matter can be found here: http://emc23.com/tutorials/34-tutorials ... me-scripts
Cool. Thanks for visiting and sharing. Your site doesn't seem well suited for discussion, however. It seems more like a traditional blog, in that it is one-to-many publishing model, instead of a many-to-many forum model. Not a complaint, just mentioning.

Post Reply