Topic: Discussions on Modding.

[[UFP]Exeter]

Let me back up and pull all my thoughts in to gain perspective.

These are decisions I made over the last 2 years.  Does not mean it i set in stone, but some things are.

I wanted a game that was highly moddable and customizable, but still fast, stable and a secure core set.  I investigated both xml and database for storing game data (non core) and decided on sql, as it is easier to work with.  And modders can have ready access to it.  From my point of view the database was both faster and easier.

I was not going to sacrifice performance to enhance modding.  This meant using heavy script like LUA for the game mechanics was out.

I wanted a secure game core.  This is portions of the game that cannot be modded.  At the same time let players build their own game core.  So I decided a secure database.  I can sore what I consider critical, but if a modder wants to they can change it.  This is for advanced modding as you can change as you can change weapons range, damage, sensor ranges etc.  Literally make the game into what you want.  To mod this requires building a new database or copying ours and renaming it and changing it.

Then a skilled modder, still works with a database, but can change the existing.  This would allow changes the graphics, specific models etc.  Nothing concerning the mechanics but can substitute graphics, models audio etc.  This also has zip files that can be modded. 

The low level mod woul be to change the zip files that contain non proprietary visual information.  This is the only moding level that does not require some database knowledge.

Database knowledge requires no SQL.  You can use a firefox plugin to open the database and then edit like a spreadsheet.  Very easy to mod.
Our proprietary assets will be stored in encrypted zip files.

We will be pro modding.  Any issues or bugs we will fix and will answer questions to assist modding.  As the highest level, I want a modder to be able to make this into a Star Wars game and there be no clue that there ever was Star Trek.

These ideas came from me thinking of a generic space game and a mod to turn it into pure trek.

I think the only way to make this more mod friendly would be to release the source of build the entire game in LUA or PYTHON etc.

 

[Starfox1701]

Ok first off I think there's a basic communication problem as I think I lack the functional vocabulary. Not your fault. So with that in mind let my ask a few questions so that not just me but others around her who lack a strong programing background can understand exactly what we are talking about.

I wanted a secure game core.  This is portions of the game that cannot be modded.  At the same time let players build their own game core.

1. My understanding is that the core is the the game engine; that all of the hardcoded stuff is here. It is my understanding that unless you decompile it and can program that this is the part of the game that cant be changed at all. So my first question is what specific elements of the game do you consider to make up the core?

So I decided a secure database.  I can sore what I consider critical, but if a modder wants to they can change it.  This is for advanced modding as you can change as you can change weapons range, damage, sensor ranges etc.  Literally make the game into what you want.  To mod this requires building a new database or copying ours and renaming it and changing it.

Then a skilled modder, still works with a database, but can change the existing.  This would allow changes the graphics, specific models etc.  Nothing concerning the mechanics but can substitute graphics, models audio etc.  This also has zip files that can be modded. 

The low level mod woul be to change the zip files that contain non proprietary visual information.  This is the only moding level that does not require some database knowledge.

Database knowledge requires no SQL.  You can use a firefox plugin to open the database and then edit like a spreadsheet.  Very easy to mod.
Our proprietary assets will be stored in encrypted zip files.

2. You keep mentioning Databases and in the above post you talk about secured and unsecured Databases. At first I though you where using the term interchangeably with Libraries, but this no longer appears to be the case. So what exactly are these Databases and what do they contain?

[[UFP]Exeter]

1. My understanding is that the core is the the game engine; that all of the hardcoded stuff is here. It is my understanding that unless you decompile it and can program that this is the part of the game that cant be changed at all. So my first question is what specific elements of the game do you consider to make up the core?

I consider the core to be the game engine (physics, graphics, network, audio etc) and the combat system.  The parameters that drive the combat system are in the encrypted database.

2. You keep mentioning Databases and in the above post you talk about secured and unsecured Databases. At first I though you where using the term interchangeably with Libraries, but this no longer appears to be the case. So what exactly are these Databases and what do they contain?

There is an encrypted and a non encrypted database.  The encrypted database holds parameters for the combat system which includes ship and weapon specs etc.  But only the specs crucial to the combat system.

all other data is stored in an unecrypted database. 

[Starfox1701]

1. My understanding is that the core is the the game engine; that all of the hardcoded stuff is here. It is my understanding that unless you decompile it and can program that this is the part of the game that cant be changed at all. So my first question is what specific elements of the game do you consider to make up the core?

I consider the core to be the game engine (physics, graphics, network, audio etc) and the combat system.  The parameters that drive the combat system are in the encrypted database.

So you consider  all the script and .CFG files to be part of the core as well as all models, textures, and sound effects?

2. You keep mentioning Databases and in the above post you talk about secured and unsecured Databases. At first I though you where using the term interchangeably with Libraries, but this no longer appears to be the case. So what exactly are these Databases and what do they contain?

There is an encrypted and a non encrypted database.  The encrypted database holds parameters for the combat system which includes ship and weapon specs etc.  But only the specs crucial to the combat system. All other data is stored in an unencrypted database.

Ok now we are getting somewhere; What ships and weapon specs do you think need to be encrypted? From my modding experience I wouldn't expect that any specific ship or weapon scripts would need encrypted for example the Galaxy script or the type X phaser script or the klingon light disruptor script. There are a lot of different kinds of script and data files that effect the way ships conduct battles that should be open to easy moding. Tinkering with the various script files is the basic entry level of moding in my experience.

Been thinking and it seams a good bit of your concern about what is modable is that others will break there copy of the game and then ask you to fix their mistakes. I might have a simple solution. Have you ever played a game with a mod manager as part of the engine? In the event you or others haven't I will explain a bit. This allows the adding and running of mods without overwriting any of the stock files. The mods are placed in a Mod directory and accessed from the options menu ingame. This allows the player to keep everything organized and mod things without fear of bricking the whole install and it makes debugging a mod easier and at worst case only a single mod in the folder would be rendered inoperable.  I have encountered this system in Both the 40k DoW game and the SW EAW game. The Fleet Ops team has also retrofitted one into their version of Armada 2.

[[UFP]Exeter]

So you consider  all the script and .CFG files to be part of the core as well as all models, textures, and sound effects?

I mean the game engine which is composed of the graphics engine, network engine, physics engine etc.  Assets such as models, textures, scripts .cfg etc are not part of the core, although they are used by the core.

Just trying to clarify my statements, the game engine is the core along with the combat system.  The core utilizes data, but the data is not all considered core data.  Some of the data is encrypted.

Ok now we are getting somewhere; What ships and weapon specs do you think need to be encrypted? From my modding experience I wouldn't expect that any specific ship or weapon scripts would need encrypted for example the Galaxy script or the type X phaser script or the klingon light disruptor script. There are a lot of different kinds of script and data files that effect the way ships conduct battles that should be open to easy moding. Tinkering with the various script files is the basic entry level of moding in my experience.

Using you example, I would consider the maximum range and the maximum damage done to be data for encryption.

For ships, the actual mass for example does not need to be encrypted, but the maximum mass for this class of ship across all races could be.  Also, an impulse engine, the maximum thrust for a lower level impuse engine could be encrypted but the exact thrust of a Romulan or Klingon does not have to be.

I want a game that is playable out of the box, that is balance and allows modders to change much of it.  Ship A fires Wep X and ship B at range R with shields S an hull H.   All have limits based on its class.  This is critical as we allow upgrading of gear.  If you want to put a disruptor on a federation ship I do not care.  The core would be checking ranges, then determining if it hit, considering other forces (a black hole will bend phaser beams) then damage calculations.  Most of the data used is not encrypted.  Only the maximum limits are.  Like Type X phasers on a sabre.

I am no preventing creation of weapons or modding them, but each weapon will have energy and mass requirements along with range and damage.  A weapon small enough for a shuttle, minimal energy requirements, with the range of a phaser X and the damage of a phaser X will not be acceptable by the game.

[Starfox1701]

OK 2 more questions

1. What would be involved in modding these encrypted files; ie will they be accessible by the player?

2. Why do these theoretical maximums need to be encrypted if the mod manager is going to preserve all the stock files in there entirety?

[[UFP]Exeter]

1. What would be involved in modding these encrypted files; ie will they be accessible by the player?

The encrypted files themselves cannot be changed but a modder.  However, a modder can create their own file with their own data to replace the encrypted.  I intend to add the ability to copy our encrypted and they can use it and change it all they want.  In the configuration the player can change the database name they use.  They could have hundreds.  However, network play would require the original encrypted file.

2. Why do these theoretical maximums need to be encrypted if the mod manager is going to preserve all the stock files in there entirety?

At the time I planned this there was no Mod Manager.  Second the Mod Manager should be to control the models.  The theoretical maximums and other data are part of the game system.  To put critical game desing or parameters in the hands of the mod manager is something I have problems with.   Following the tenants of OOP, the mod manager should be to control aspects relating to the model.  Physical characteristics I do not view as part of the model.  Not saying I am correct, just the way I view it.

[Starfox1701]

1. What would be involved in modding these encrypted files; ie will they be accessible by the player?

The encrypted files themselves cannot be changed but a modder.  However, a modder can create their own file with their own data to replace the encrypted.  I intend to add the ability to copy our encrypted and they can use it and change it all they want.  In the configuration the player can change the database name they use.  They could have hundreds.  However, network play would require the original encrypted file.

If I as the moder can replace the encrypted files anyway why bother encrypting them? From what you have said it looks like extra work that has no benefit for anyone. If replacing them does not break the game, ie cause it to crash, what is the logical reason for the encryption?

2. Why do these theoretical maximums need to be encrypted if the mod manager is going to preserve all the stock files in there entirety?

At the time I planned this there was no Mod Manager.  Second the Mod Manager should be to control the models.  The theoretical maximums and other data are part of the game system.  To put critical game desing or parameters in the hands of the mod manager is something I have problems with.   Following the tenants of OOP, the mod manager should be to control aspects relating to the model.  Physical characteristics I do not view as part of the model.  Not saying I am correct, just the way I view it.

The mod manager is far more useful then just swapping models. Broadly there are at least 4 benefits to having a mod manager ingame as part of the engine.

1. Security in the form of preventing the moder from accidentally rendering the game nonfunctional. If the moder is making lots of changes and not testing each new ship and script file this prevents him of her from having to reinstall the whole game and possibly losing saved games and other mode in different folders. Only that individual mod is broken and all of the stock files still exist should they need to copy them and put them in the mod to restore function and fix their mistake.

2. It preserves multiplayer and online gaming ability because all of the stock files are present in an unaltered state. it also makes it easier for players to use mods for multyplayer as the installation process is simplified in a mod manager greatly reducing errors and the sink issues they cause.

3. It lets individuals easily organize the mods they are using all in the same install while preserving the functionality of all the mods and the stock game. This saves tons of Hard Drive space and limits fragmentaion of the Hard Drive. It also makes backing up a particularly heavily moded install simpler.

4. It provides a single common framework for the construction and release of mods. This will just benefit the whole community as it is both simpler to learn how to make these mods but almost foolproof to install them.

So you can see a mod manager is far more then just a model swapper.

[[UFP]Exeter]

If I as the moder can replace the encrypted files anyway why bother encrypting them? From what you have said it looks like extra work that has no benefit for anyone. If replacing them does not break the game, ie cause it to crash, what is the logical reason for the encryption?

Two reasons.  Some of the information I have obtained we can use freely but it requires encryption to prevent use in other endeavors.  Also, it prevents editing of the games database.  I do not mind if they make their own, but they cannot edit ours.

The mod manager is far more useful then just swapping models. Broadly there are at least 4 benefits to having a mod manager ingame as part of the engine.

Major misunderstanding here.  I thought we were talking an external mod manager similar to what NanoFX does for Excalibur.  If it is part of the game engine, then I have no issues as I can store the information how I please as it will have access to databases, scripts and models.  I already planned an editor to load user defined models and textures into the database to be used by the game.

1. Security in the form of preventing the moder from accidentally rendering the game nonfunctional. If the moder is making lots of changes and not testing each new ship and script file this prevents him of her from having to reinstall the whole game and possibly losing saved games and other mode in different folders. Only that individual mod is broken and all of the stock files still exist should they need to copy them and put them in the mod to restore function and fix their mistake.

Agreed this is not our problem.  However, I would like players with the same mod level to be able to multiplayer with it.  So what if we left nearly all the data open, and put a method to scan and mark a database as "safe" for mutiplayer with a version number, encrypted, that confirms everyone in multiplayer is using the same database?

[Starfox1701]

If I as the moder can replace the encrypted files anyway why bother encrypting them? From what you have said it looks like extra work that has no benefit for anyone. If replacing them does not break the game, ie cause it to crash, what is the logical reason for the encryption?

Two reasons.  Some of the information I have obtained we can use freely but it requires encryption to prevent use in other endeavors.  Also, it prevents editing of the games database.  I do not mind if they make their own, but they cannot edit ours.

What specific scripted information falls into that category of 3rd party stuff can't be used by others?

 Let me be clear here, 99.9% of modders out there don't have the skill set to rewrite program code so they won't want access to that. What they want is the scripted stuff. For example the program code that tells a phaser beam to start at the firing Hp and stretch out to the target and move from Hp to Hp on both the firing and target objects. This is the core behavior of that weapon class and is something that modders shouldn't need access to. Everything else about phasers and other phaser like weapons should be scriptable and therefore should be open to the modders.

As far as open sourcing the program code after release I think that's a good idea. It will facilitate things like server packages and the like as well as the making of 3rd party plugins for DCCs that fall outside of out construction and support pipeline. It will also help with hardware specific bugs that crop up because they where outside of our test pipeline. Will the 3rd part stuff make that possible?

The mod manager is far more useful then just swapping models. Broadly there are at least 4 benefits to having a mod manager ingame as part of the engine.

Major misunderstanding here.  I thought we were talking an external mod manager similar to what NanoFX does for Excalibur.  If it is part of the game engine, then I have no issues as I can store the information how I please as it will have access to databases, scripts and models.  I already planned an editor to load user defined models and textures into the database to be used by the game.

I don't think the mod manager will have to store anything. All the mod data would be in the there own folders in the Mod folder inside the game on the Hard Drive. From a programing standpoint, as I understand it, the manager just tells the engine to use the mod files instead of their stock counterparts; filling in any blanks in the mod with the needed files from the stock install during the loading of the game. In this way each mod is a separate set of databases that don't effect the integrity of the stock databases at all. I guess that would make it some kind of asset exchange protocol?

1. Security in the form of preventing the moder from accidentally rendering the game nonfunctional. If the moder is making lots of changes and not testing each new ship and script file this prevents him of her from having to reinstall the whole game and possibly losing saved games and other mode in different folders. Only that individual mod is broken and all of the stock files still exist should they need to copy them and put them in the mod to restore function and fix their mistake.

Agreed this is not our problem.  However, I would like players with the same mod level to be able to multiplayer with it.  So what if we left nearly all the data open, and put a method to scan and mark a database as "safe" for mutiplayer with a version number, encrypted, that confirms everyone in multiplayer is using the same database?

Well traditionally it is up to the modder to test a mod for multiplayer compatibility. If they break it it is their job to fix it or tell the community it's broke and not to use it. They can ask the community for help in fixing it but as the game creators we wouldn't be under any special obligation to fix it for them. However so long all the players are using 100% identical mod installs and any additional scripted elements are stable mutiplayer compatibility should not be broken. One thing we can do to facilitate this is include an info file format for them to use. Have it include things like the mod name and version number as well as the game version it is designed to work with. The file might also include script lines to tell the mod manger whether it is a stand alone mod or whether it uses files from other Parent mods or the stock game. Here's an example from Fleet Ops of setting up such a file.
http://guide.fleetops.net/guide/modding/modmodules/settings

[[UFP]Exeter]

What specific scripted information falls into that category of 3rd party stuff can't be used by others?

 Let me be clear here, 99.9% of modders out there don't have the skill set to rewrite program code so they won't want access to that. What they want is the scripted stuff. For example the program code that tells a phaser beam to start at the firing Hp and stretch out to the target and move from Hp to Hp on both the firing and target objects. This is the core behavior of that weapon class and is something that modders shouldn't need access to. Everything else about phasers and other phaser like weapons should be scriptable and therefore should be open to the modders.

As far as open sourcing the program code after release I think that's a good idea. It will facilitate things like server packages and the like as well as the making of 3rd party plugins for DCCs that fall outside of out construction and support pipeline. It will also help with hardware specific bugs that crop up because they where outside of our test pipeline. Will the 3rd part stuff make that possible?

Modders would like everything scripted    A database is so much easier to build and maintain.   So if we used scripts, how do we make sure multiplayer is not affected?  As for scripting there are two ways, using the file to only contain data like and xml file, of to actually program and control some of the games functions, such as LUA.  I know some XML and no LUA.

I don't think the mod manager will have to store anything. All the mod data would be in the there own folders in the Mod folder inside the game on the Hard Drive. From a programing standpoint, as I understand it, the manager just tells the engine to use the mod files instead of their stock counterparts; filling in any blanks in the mod with the needed files from the stock install during the loading of the game. In this way each mod is a separate set of databases that don't effect the integrity of the stock databases at all. I guess that would make it some kind of asset exchange protocol?

This idea actually automates what I conceived of and was a manual process. 

From this thread I am leaning toward this:
Game data in an NON encryoted database.   Other data may be stored in scripts.  Media will be stored in zip files.   Depending on the media, it may be encrypted.  The encryption is for assets that are copyright that we have permission to use.  There will be a file (or database) that specifies the locations of assets,,etc.  This can be modified by the modder

We can do a mod manager to assist in this, but that can be done later

[Captain Adam]

.

[[UFP]Exeter]

That is what we are discussing.  I sort wanted to keep that data safe and allow moders to copy their own.  Also read the thread on modelling and join in, we need more perspective by modders.

There is one limitation, the amount of scripting, I do not have the skill or desire to either learn or write a game in LUA.

[Captain Adam]

.

[Starfox1701]

What specific scripted information falls into that category of 3rd party stuff can't be used by others?

 Let me be clear here, 99.9% of modders out there don't have the skill set to rewrite program code so they won't want access to that. What they want is the scripted stuff. For example the program code that tells a phaser beam to start at the firing Hp and stretch out to the target and move from Hp to Hp on both the firing and target objects. This is the core behavior of that weapon class and is something that modders shouldn't need access to. Everything else about phasers and other phaser like weapons should be scriptable and therefore should be open to the modders.

As far as open sourcing the program code after release I think that's a good idea. It will facilitate things like server packages and the like as well as the making of 3rd party plugins for DCCs that fall outside of out construction and support pipeline. It will also help with hardware specific bugs that crop up because they where outside of our test pipeline. Will the 3rd part stuff make that possible?

Modders would like everything scripted    A database is so much easier to build and maintain.   So if we used scripts, how do we make sure multiplayer is not affected?  As for scripting there are two ways, using the file to only contain data like and xml file, of to actually program and control some of the games functions, such as LUA.  I know some XML and no LUA.

XML is not intended as a scripting language; Lua is. From the bit of research I did today the only other language that competes with it is Python. From what if found Python has the advantage of lots of libraries out of the box so to speak and tons of specific commands so there's pretty much nothing it can't do. Down side is the Python is very weighty and therefore can add both to the overall size of the game and to load times. Scripting will also tend to be heavy. Also Python is a programing language in it;s own right and can be a pain to integrate with C++ as it was never intended to be embedded as a scripting language. Python just received a version upgrade and may also have stability issues but I was unable to confirm that. Lua on the other hand was created from the start as an embeddable scripting language. It is light weight and fast and suppose to be very easy both to learn and master. While both are free too use and have active communities that support them but Lau appears to have the better engineers and even people who didn't like it admitted it had a long track record of stability. Down side is that Lua lacks the out of the box library support of Python. They have to be rounded up or converted from C++. Functionally Lua is suppose to be the equal of Python but lacks vast array of discrete commands. Lua is far more popular in the gaming industry having been used for a lot of well known titles
http://en.wikipedia.org/wiki/Category:Lua-scripted_video_games   http://www.lua.org/
Python is also used but not quite as often
http://wiki.python.org/moin/PythonGames
It should be noted the Bridge Commander was a Python scripted game. I have some Python experience but think we should at least look at Lua before deciding.

I don't think the mod manager will have to store anything. All the mod data would be in the there own folders in the Mod folder inside the game on the Hard Drive. From a programing standpoint, as I understand it, the manager just tells the engine to use the mod files instead of their stock counterparts; filling in any blanks in the mod with the needed files from the stock install during the loading of the game. In this way each mod is a separate set of databases that don't effect the integrity of the stock databases at all. I guess that would make it some kind of asset exchange protocol?

This idea actually automates what I conceived of and was a manual process. 

From this thread I am leaning toward this:
Game data in an NON encryoted database.   Other data may be stored in scripts.  Media will be stored in zip files.   Depending on the media, it may be encrypted.  The encryption is for assets that are copyright that we have permission to use.  There will be a file (or database) that specifies the locations of assets,,etc.  This can be modified by the modder

We can do a mod manager to assist in this, but that can be done later

Yea the Mod manager is probably the last tool that will need to be done. We will definitely need the both model property editor and map editor/mission scripter first.

As much as I like playing multiplayer, I do like modding and playing with the AI.
Will I be able to copy the existing weapons, make new, change output damage, ranges, etc...

Adam

That's the plan, We haven't settled on the scripting language yet and there is still a discussion on how much to script. I am personally on the side of scripting as much as possible with the various asset features. Finding the right balance will be important.

That is what we are discussing.  I sort wanted to keep that data safe and allow moders to copy their own.  Also read the thread on modelling and join in, we need more perspective by modders.

There is one limitation, the amount of scripting, I do not have the skill or desire to either learn or write a game in LUA.

We don't actually know how much scripting we will need yet. If we can link files in trees that will simplify the process as we don't then have to re-script things over and over.

That's a big one for me... To be able to copy weapons profiles and change certain characteristics would be nice. I don't want to be limited in that aspect. A good game shoukd allow modders to make/copy a weapon, alter it's textures, ranges, damage output, sound effect, etc... Should be made easy too.
Will we be able to target subsystems? I know, random question lol. Sorry.

There should also be codes in place to make certain weapons behave differently, for instance like when certain weapons dampen shields, disable weapons, dampen engines, etc...

Hmmm. Coming from a modders perspective I think We should make this as open as possible and not restrictive.

I think this kind of minutia is part of what defines the SFC style of games. Right now with the TNG focus for the first release there really isn't a wide variety of different weapons in the 5 main races. Besides the ones that have to be there I'd also like to see some version of missiles, hellbore type weapons, ESGs, and web spinners/casters in the stock release. I would also consider some of the A2 and KA special weapons for inclusion either in stock or as part of expansions.

[Starfox1701]

Did you ever look into these 2 choices for a scripting language?

[[UFP]Exeter]

I looked into Python, LUA and Angelscript.

I had planned on scripting for missions, to control how the missions run.

For storage of data, not necessary for a scripting language, we can use XML for that.  Or a database.

[Starfox1701]

OK the way I comprehend your answer is that you are going to do an XML framework for databases but what language are we going to use for the files in these databases?

Ok when I talk about scripting I'm talking about things like the file that tells the game which sound and sprite/particle effect/model a phaser uses and things like beam scale and speed. I also want to make it clear that the scripted files are something I don't expect you to do all by yourself. Like I said earlier scripting is widely considered entry level modding. With that in mind I personally want to be involved in doing them as well documenting all the different command variations so that 1. We have it to speed the process and 2. for modders to use and learn from so they make fewer mistakes. I'm sure I won't be the only one.

[[UFP]Exeter]

It will either be 1. XML with ZIP files   or 2.  Database.  Depending on how we go with the scripting language.  I may.


As for scripting, what we do can be flexible.  But we do need one.  Now which one?  My vote is for AngelScript.  Because it interfaces easily, been around for a while and the syntax is similar to C++, so easier for me to use, learn and assist.  I have no problems with LUA, would be nice to have someone with experience in that.  But I think Python is not a good choice.

[Starfox1701]

I'm leaning towards Lua as it free to use, is suppose to be easy to embed in a C++ program, has a long history of stability, is considered readable(ie people who don't know programing can readily understand the syntax), has an active community and is still supported, and is the one most commonly used for video games scripting. It means learning something new but I think it is the best tool for the this job.

[Captain Adam]

.

[Starfox1701]

I suppose too since Homeworld 2, Dawn of War and Empire at War have such big modding communities we can see if they have any scripting tools that might give us ideas on how to speed this up.

Going Out of town, be back tomorrow. If any one has Lua experience please chime in.

[[UFP]Exeter]

LUA is not that easy to embed a parser etc into c++ projects.  Still looking at it.  AngetScript tool 5 minutes.  But sounds like LUA may be the best choice.  And with real game samples we can ge an idea of the best way to do this.