Page 1 of 1

Development Update...

Posted: Fri Aug 11, 2017 05:36 UTC
by TheMightyDude
Hey All

Sorry I haven't been around much, been having some RL issues to do with my health and other stuff, so I had to concentrate on those first.

Thank you to a few users that have emailed me asking if everything was ok, it was appreciated.

I have some answers to some of your questions which I will list later in this post.

To-do with my health...
I have a lung condition that affects 1 in ever 1,000 people to do with the air ways in the longs swelling up with mucus making it hard to breath.
Over the last several years I have been put on various medications in which some helped a little and a few did nothing, and then there was all the side affects to them which wasn't nice I tell you.

The last several months they finally diagnosed me with this conditions where I have about 90% of the symptoms, so its a very good chance its that, sadly there is no cure for that but only medication to slow or stop it getting worse and at the same time makes things bearable, which is all good news in my book.

Now BNT...

A lot have asked would it be the same as current BNT.
Now here is the issue, I would like to keep it the same as a browser based game, I really would, but like I have said before the way browsers work with HTML etc if its a Request and Response type of game, so for stuff to be sent to the client (in this case a browser) the client needs to do a request to be told something has happened etc.

This is why we only get notifications of an in game message when you do something in the game, you have basically done the request.

So we are very limited to what we can add and this was why I really wanted to move away from browser based usage and over to a desktop application.

But I was really thinking about this the other day while I was doing some code to-do with Octrees which will be used for Memory Data Storage to lower the Database usage etc.
Anyhow I was looking into the road map for the Game Engine that I am currently using and they are improving WebGL which runs in a browser, granted it was very slow on most browsers in the older released versions of the engine, but it seems that it is getting better.

So I thought I would do the game as I was before as a Desktop Application but also add in support for it to cross-platform compatible for WebGL as well ready for if I release the WebGL version later on when its a lot better.

You said you will be using a 3D Game Engine, so will it be a 3D or 2D game?
Well I did plan to have it as a 3D version with 3D Space Ships, Space Stations and players being able to fly about in the universe.
But I really need to be realistic here, this hopeful thinking on my part was just silly, but there is nothing saying this might not happen later down the line in the future.

No, basically it "should" look and play exactly how it does now but without all the browser exploits like multi-tabs etc. which "I know" some are still currently using.

The only difference should really be its being run in a Desktop Game Client instead of a browser, it will still be a point and click type of game along with some extra stuff like Server to Client Notification.

Extra features...
Server to Client Notification is as it says messages that the server sends to the game client which is used for new future features I want to add later on.
This is also going to be used for in-game mail / messages etc, this can be sent across servers which may or may not be run be us.
So Player A is on our game server playing and sees a friend Player B join a 3rd part server running this game, so Player A and Player B can send in-game mail / messages to and from each other.

There will also be a Friends List feature where friends can always see when they login or logout and also what game server they are playing on.

There is way more features that could be added later on.

What about being able to play the game on my browser?
Well like I have already said at the start of this post, I still would like to have the game to be able to be also played via the browser, but I don't think the browsers are there yet, granted there is WebGL and HTML5 but until all browser handles all aspects of those equally, stable and fast I will put that on the back burner and see how it goes over the next year or two.

So for now I am not saying wont be released as a browser game, but more towards, not at the moment.

When the desktop version is complete will there be server downloads available for us to setup and run.
Not at first, but eventually yes there will.
I just wanted to make sure everything was working right before I mad it available to the public.

Ok, so the server will be available later on, but what will I need to run it on my server.
Well I an planning for the Server to be released as compiled code for a Linux Server, MySQL (at first, others later on) will be needed for the database, PHP and Apache if I add an Armoury / Stats Page.

Apart from that, it should just be the matter of uploading the server files to the server, add cron to run it 5 mins after server start-up and that should be it.

Also the Game Server should be run as its own user and NOT root.

People that want to run their own server will need to create an account on our servers like normal players would and also request for a Server API Key which are unique per Server, so if you run 7 Game Servers you will need to request for 7 API Keys.

Also 1 or more ports will be required to be opened on your firewall for players to be able to connect to it.

Also its one Game Server per IP by default at the moment, but I have been looking into allowing it to use different ports, but that can be addressed later on.

What about creating accounts every time the game is reset? and how is this handled when 3rd part servers are allowed?
That is a very good question...
Game Servers 3rd party and our ones will not store any account information, they will only store game required information in their databases.
All user account information will be stored on our network / login servers etc.
All Game Servers including ours will all request a unique hash / UUID based off the users Account ID and the servers ID that they are playing on.
The connecting Client (already logged in on our network) connects up to a Game Server and tells it its Connection ID (random and changes every login) the Game Server then uses that ID along with its API Key, ServerID and the servers encrypted Password to our network and at our end we lookup the required users info to validate its all valid and then generate a UUID based from both the Clients Account ID (Int64) and ServerID (Int64) and send that back to the requesting Game Server.

The Game Server will now use that UUID to represent that User every time they return and play on that server.

Also all communication between the Clients and Servers and also the Game Server to our Network are all encrypted using RSA Key Pairs and also uses SSL/TLS Secure Connections a bit like how HTTPS works.

Well there is loads more, but these were the most asked questions.

Re: Development Update...

Posted: Sun Dec 10, 2017 09:47 UTC
by TheMightyDude
Well I had a slight setback, due to issues on Windows 10 (i.e. my Main System) which run several Virtual Servers (i.e. Development Servers) I had to reinstall Windows 10, I backed up everything apart from the Virtual Machines.

So after 36 hours once I got it back to a state where everything was installed I noticed there was no Virtual Machines Folder, that's when it dawned on me that I hadn't backed that up.

While that was a rather painful mistake it wasn't that bad, granted I lost all server side code, but some of the code (i.e. generic code etc) was also used on the client side code.
Also I had worked out better more efficient ways to do some of the server code which would of been a pain to do with the other code without gutting stuff.

So looking at what was lost, I restarted the server side code once more from the ground up, splitting up loads of classes and rearranging them more better.

I will be trying to keep the code arranged so that later on if the game takes off and more players come to play, it "should" be easy to rollout a newer version in the form of a Node Server.

At first it would be just the one Server Node being used to run the whole Game Server and later on if needed I could add more of these Node to support more connected players.

At this point I know my last network code was able to handle 5K user connection sending and receiving random network messages.
So if I aim for half that we should be fine, its not like we will hit 2,500 connected users right away now is it :P

So all the changes...
Server code now uses a revamped version of my Thread Manager Class, so its more better handling multiple threads.
Network along with the Read and Write Message Queues are now using this new version of Thread Manager so I can now create Threads with additional Support Threads.

The networking code still needs some work.

AI Manager (not started yet) will also be using the Thread Manager so that will be running in the background and if done right they will act like players, well they will have the same limitations the players have, AI Manager will be updating every 1mS but each AI will take their turn every X Seconds.

I am currently working on the code that does the creation of the Universe as a whole (Galaxay Clusters, Systems, Sectors, Planets, Ports, Links, Zones etc), I had it create 500,000 current BNT sectors in less thaan 10mS (guessing) it was that instant.

The tricky part for getting it to talk to the MySQL Server, so I had to create my own class to do all that.
Once thats fully working I will be working on the Storage Memory where all database information will get loaded and stored into and is what the Game Server Node will use when players are playing, this is done so that the database isn't hammered all the time, plus RAM is faster than HDD / SSD.

I will be using Octrees / Quadtrees to store the links to the loaded data which will be stored in a form Hashed Lists etc.

I have also got a screenshot of the Server Node running, its nothing special, but once I get to a stage where the client can login etc I will start doing videos etc.
Thumb imageImage of the Server Node Console,nothing special at the moment...
As you can see its very basic, but it shows the starting up info with all the threads etc.
It can handle connections, it just won't know what to do with all the network messages etc.

Well that's it for now.

Re: Development Update...

Posted: Fri Dec 22, 2017 17:40 UTC
by TheMightyDude
This is a very short update...

I have not long started on re-structuring the database tables.

The old ships table will change a lot, it will be split up into several tables:
Accounts, Characters, Presets, Ship, Ship Cargo, Ship Devices etc.

This should open us up ready for more resources that we can trade and more Presets etc.

Re: Development Update...

Posted: Tue Dec 26, 2017 16:37 UTC
by TheMightyDude
Just noticed a bug in Create Universe when I was coding my new version, the issue is where it creates the Special Ports.

Code: Select all

$sql_query=$db->Execute("SELECT sector_id FROM {$db->prefix}universe WHERE port_type='none' ORDER BY RAND() DESC LIMIT $spp"); 
This is done right after we allocate all the Federation Sectors which are set to ZoneID 2.

This issue is when we setup all the Special Ports we are not checking the ZoneID, so we run the risk of loosing federation sectors due to when allocating the special ports we set their zone id to 3 (i.e. Free-Trade space).

I did loads of tests and several times it over wrote federation sectors, in one test it lost 7 out of the 25 federation sectors which is bad.
The fix is the following:

Code: Select all

$sql_query=$db->Execute("SELECT sector_id FROM {$db->prefix}universe WHERE port_type='none' AND zone_id = 1 ORDER BY RAND() DESC LIMIT $spp"); 
This way it will only allocate the unallocated sectors, which is what we want.

That aside development is going rather well, health hasn't been that bad so I have spent several hours doing stuff yesterday, currently working on the creation of the database and a Storage Manager where all the database is loaded into when the dedicated server starts up.

This is done to reduce almost all SQL Database Queries when players are playing, so while players are playing they are only accessing the information stored in RAM and RAM is a hell of a lot faster than HDD and even SSD's.

I am starting to like Linq when I need to query loaded data in RAM :)
A similar version for the Special Ports Code:

Code: Select all

storage.sectors.Where(
    s=>s.port_type == Sector.PortTypes.none && s.zone_id == 1
).OrderBy(
    s=>rand.Next()
).Take<Sector>(spp)
I just tried a universe size of 2.5 Mil Sectors and it did it in about 1 to 2 seconds on a virtual machine with 2 CPU Cores and 1GB of RAM, now doing this in PHP would of taken a lot longer and probably crashed or timed out.

Its not that we would ever have a Universe of that size :P

Table Creation completed.

Sector Creation.
 - Creating 2500000 Empty Sectors.
 - Creating Sol Sector 0
 - Creating Alpha Centauri Sector 1
 - Allocating 12500 Federation Sectors.

Port Creation.
 - Allocating 25000 Free-Trade Special Ports
 - Allocating 375000 Ore Ports
 - Allocating 250000 Organics Ports
 - Allocating 375000 Goods Ports
 - Allocating 250000 Energy Ports

Creating 250000 Planets in Random Sectors.


But saying that, the new code isn't completed yet.

More stuff to come later.

Re: Development Update...

Posted: Wed Jan 17, 2018 08:51 UTC
by TheMightyDude
Another update...

I have been working on the storage side of the Dedicated Server code.

There are at least two ways to do this and follows:
  • Have the Server do the SQL Queries when it needs to like how Apache does it now.
    This the lowest RAM Usage, but if loads of players start playing it starts to hammer the Database with SQL Queries.

  • Have the Server load everything into Memory and only update the data in memory and save the changes to the database every X mins.
    This has a much higher RAM usage depending on the size of the universe, a 2.5Mil Sector Universe takes up around 1GB of RAM with a high spike of 2GB, but has a much lower usage on the database.
I am aiming for the second route its a lot harder to implement than the first, its not that we will have a universe with 2.5Mil Sectors to play in :P

A universe of 35,000 sectors is currently taking about 49MB of RAM.
And a Universe of 2,500 sectors is currently taking about 35MB of RAM.

The two above memory usage will change as player play the game by creating planets, links, traderoutes and so on.

The way this is suppose to work is as follows...
  1. The Server starts up and gets all the universe information (Sectors, Planets, Links, Players and so on) and stores it all in RAM.
  2. A new Player logs in and their information is then loaded from the database and stored into RAM.
  3. Now everything that happens in game will only update the data located in RAM and not the database.
  4. Every X Time (say every 5 mins it will update the database with all that has changed.
  5. When a player logs out it will update that players information in RAM, however the players information will remain loaded in RAM.
I could have it force the player to dock at a Space Station which will cost the player credits to park, but at least the player will be safe over night.

More information to come.

Re: Development Update...

Posted: Thu Feb 08, 2018 17:24 UTC
by TheMightyDude
Sorry not been about much as late, RL issues as usual, loads of medical appointments etc.

Just a very quick update for the Dedicated Server which can now create the database.



In the above video clip you will see a Universe of 3,500 Sectors created instantly and then a Universe of 100,000 Sectors which will take a little longer to create.

And yeah I am aware I haven't done the Ports part of the code yet, that was due to where I changed the code to be more modular.

You normally wouldn't run a BNT Game with 100,000 Sectors, but I thought I would show it as a comparison.

Probably next week I will start to clean up the code and remove redundant code.

I will then look into the loading of the database into memory parts of the code.

Re: Development Update...

Posted: Sat Feb 10, 2018 21:03 UTC
by thekabal
*LOVE* the Ansi art!!

Re: Development Update...

Posted: Sat Feb 10, 2018 21:32 UTC
by TheMightyDude
thekabal wrote:*LOVE* the Ansi art!!
I know right, it looks soo nice :)

Re: Development Update...

Posted: Wed Dec 12, 2018 22:23 UTC
by TheMightyDude
Well I haven't posted much as late, been ill with several Lung Infections, but had a bit of free time so I thought I would do some programming :)

Until now I have been coding and testing on a Windows PC using C# which was working rather well, well until I moved my framework over to Linux, created the Makefile and all the required references and files etc.

This was when I noticed that the valid Certificate which we use on our sites which works fine, just failed outright in the Server Code.

Basically they were just failing to validate when run on Linux, this is due to Linux will only validate certificates using CRL and not OCSP and Where Let's Encrypt don't add that Extension on their issued certs its fails on Linux when using Mono and I have to use Mono due to using C# and I am using sslStream over TCP so that was just failing and crashing the server.

Did I say this all works on Windows LOL.

So where I need a secure way for users to log in I then thought I would try WebRequest / WebClient to connect to a login page over HTTPS, you would think surely this would work, sadly nope, also fails in Mono on Linux.

Now after several very late nights and I mean late, I was about to just give up on the whole idea, then I thought of converting the Certificate into a PKCS12 format containing the actual cert and its private key, the issuer chain cert and export it as a PKCS12 pfx file and then tried the server code with that.
And guess what, it now works fine, sure its not ideal having to do these extra steps every 3 Months, but at least I have a working sslStream over TCP which is good.

I am still not sure how well the Unity Engine works on Linux for a Game Client and what requirement it will require to run, at the end of the dame nobody is going to download the complete mono runtime just to play a game, like I said I am not too sure how well the Unity Engine will work, so I cannot say, I am hoping I don't need to install Mono on Linux for the Unity Engine to work.

So if it becomes too much of an issue getting the Game Client working on Linux and MacOS I might drop support for those two platform for now.

So this is where I am now at the moment.