[DONE] Clean up, Part Two (directory structure)

[StefanP.MUC]

Because the other Clean up thread is getting very long and had originally some other foci, I start a new one, specifically about the directory structure. It includes the last two remaining tasks from the old thread.

First task: Collect other opinions and/or (dis)agreements.
Second task: Make a decision.
Third task: Realize.

My suggested directory structure for development (git):

{l Code}: {l Select All Code}

ODroot/
|- .gitignore
|- README
|- cmake/
    |- modules/
        |- *.cmake
    |- config/
        |- *.cfg.*
        |- *.in
|- dependencies/
    |- angelscript/
    |- tinygettext/
    |- dirent.h
|- docs/
    |- Doxyfile
    |- TODO.txt (really needed?)
    |- *.dia
    |- *.png
    |- *[otherdocfile] (licences, etcs)
|- source/
    |- *.h
    |- *.cpp
    |- CMakeLists.txt


And for the release version I suggest this:

{l Code}: {l Select All Code}

ODroot/
|- graphics/
    |- models.zip
    |- scripts.zip
    |- shader.zip
    |- textures.zip
|- gui/
|- levels/
|- music/
|- plugins/ (Ogre plugins on Windows)
|- sound/
|- [binary files]


[oln]

Looks good. We might want to consider making some folders inside the sources as well, as the number of source files is getting large.

[StefanP.MUC]

I'm away for ~2 days now. If after this time are no major disagreements or new opinions, I'll see to changing the structure to the suggested one on the weekend.

I'm not sure about the source/ sub-structure. Basically I woul agree, but I think we don't have enough source files. The GUI is only one .cpp and one .h, same for ResourceManager, same for Music and Sound and so on. Not sure how this should be ordered and split up without ending with twenty two-file-subolders or so.

[StefanP.MUC]

Quick question: why is the "Documentation" folder in the .gitignore file?

[StefanP.MUC]

Just pushed the changes to the git repo structure. Please check if everything still works (for me it works on MingW with Eclipse).

Tomorow I'll have a look at the release (media) structure.

[oln]

Will test in a bit.
The documentation folder shouldn't be in .gitignore, maybe someone meant to add the sub-folder that would be created by doxygen when generating documentation.

[StefanP.MUC]

Just pushed an update to .gitignore:
- Remove all doubled and outdated entries.
- group by type (folders, file types, concrete files)
- sort alphabetically

[StefanP.MUC]

I'm currently going through the game and config files, I noticed some apperently unneeded files, can someone confimr that this is right? If it is, then I'll remove them from the needed DLLs list from the wiki or respectivley remove them from git.

{l Code}: {l Select All Code}

//DLLS
CEGUIFalagardWRBase.dll (game runs without it)
CEGUISILLYImageCodec.dll (game runs without it)
Plugin_BSPSceneManager.dll (game runs without it, commented out, ever needed?)
Plugin_OctreeSceneManager.dll (game runs without it, commented out, ever needed?)
Plugin_OctreeZone.dll (game runs without it, commented out, ever needed?)
Plugin_PCZSceneManager.dll (game runs without it, commented out, ever needed?)
SILLY.dll (game runs without it)
libgcc_s_dw2-1.dll (game runs without it)
libstdc++-6.dll (game runs without it)
openal32.dll (only needed when OpenAL is not installed already?)

//config
(we have a plugins.cfg.in that is used instead)
plugins.cfg.linux
plugins.cfg.windows


[svenskmand]

In my opinion dll's should not be in the git repo. When I build the game on windows I always copy (there is a windows built/packaging script that does all this for you) the dlls from the SDK's we use.

[StefanP.MUC]

They are not, I meant if we can remove them from the installer and from the wiki page.

[svenskmand]

Ahhh ok. To be more specific the windows build/packaging script uses these dlls:

{l Code}: {l Select All Code}

copy "%CEGUIDIR%\bin\CEGUIBase.dll" .
copy "%CEGUIDIR%\bin\CEGUIExpatParser.dll" .
copy "%CEGUIDIR%\bin\CEGUIFalagardWRBase.dll" .
copy "%CEGUIDIR%\bin\CEGUIOgreRenderer.dll" .
copy "%CEGUIDIR%\bin\CEGUISILLYImageCodec.dll" .
copy "%CEGUIDIR%\dependencies\bin\dynamic\SILLY.dll" .
copy "%OGRE_HOME%\bin\Release\cg.dll" .
copy "%OGRE_HOME%\bin\Release\OgreMain.dll" .
copy "%OGRE_HOME%\bin\Release\OgreRTShaderSystem.dll" .
copy "%OGRE_HOME%\bin\Release\Plugin_BSPSceneManager.dll" .
copy "%OGRE_HOME%\bin\Release\Plugin_CgProgramManager.dll" .
copy "%OGRE_HOME%\bin\Release\Plugin_OctreeSceneManager.dll" .
copy "%OGRE_HOME%\bin\Release\Plugin_OctreeZone.dll" .
copy "%OGRE_HOME%\bin\Release\Plugin_PCZSceneManager.dll" .
copy "%OGRE_HOME%\bin\Release\Plugin_ParticleFX.dll" .
copy "%OGRE_HOME%\bin\Release\RenderSystem_Direct3D9.dll" .
copy "%OGRE_HOME%\bin\Release\RenderSystem_GL.dll" .
copy "%OGRE_HOME%\bin\Release\libOIS.dll" .
copy "%OGRE_HOME%\bin\Release\libboost_thread-mgw45-mt-1_44.dll" .
copy "%MINGWDIR%\bin\libgcc_s_dw2-1.dll" .
copy "%SFMLDIR%\extlibs\bin\libsndfile-1.dll" .
copy "%MINGWDIR%\bin\libstdc++-6.dll" .
copy "%SFMLDIR%\extlibs\bin\openal32.dll" .
copy "..\dep\pthreads\lib\pthreadGC2.dll" .
copy "%SFMLDIR%\lib\sfml-audio.dll" .
copy "%SFMLDIR%\lib\sfml-system.dll" .


I have not checked if they are all necessary, but did you check all of these or only some?

[StefanP.MUC]

I checked only the "suspicious" ones. Meaning: The "plugin_*" ones that are commented out (Octree*, *Manager) in the plugins.cfg and the "cryptic" ones (libstdc++-6, libgcc_s_dw2-1, SILLY, CEGUISILLYImageCodec and CEGUIFalagardWRBase).

The games runs perfectly fine without them. ParticleFX is also not used currently, but since we are planning to have particle effects soon, it can stay.

edit: just pushed the commit that now the "plugins" folder for the ogre plugins on Windows. Also shortened the DLL list on the Wiki according to the explanation above.

Also did some tests with using zip files for the graphcis resources: seems to bring some startup performance (which is much better already because of the smaller textures, anyways, but zipping seems to have reduced it by some additonal hundred milliseconds). Still need to figure out the automatic zipping from cmake, before I can push this change.

[svenskmand]

Automatic zipping from cmake? Just to be clear if we are using zip files then the scripts for making the installers should make the zip files from the media folder in SVN, agreed?

[StefanP.MUC]

Yes, for shipping. But for the devs it should also be easy to get the needed media as a zip file (checkout svn/Media, checkout git, run cmake, run make -> then the game should be runnable, too).

[svenskmand]

If your doing that, then there is no reason to do the same thing in the windows and linux build/packaging script, then they can just call cmake to do it for them.

[svenskmand]

The following files ARE needed:

{l Code}: {l Select All Code}

copy "%MINGWDIR%\bin\libstdc++-6.dll" .
copy "%MINGWDIR%\bin\libgcc_s_dw2-1.dll" .


And more is probably also need since I get this error (attached).

[oln]

The following files ARE needed:

{l Code}: {l Select All Code}

copy "%MINGWDIR%\bin\libstdc++-6.dll" .
copy "%MINGWDIR%\bin\libgcc_s_dw2-1.dll" .


And more is probably also need since I get this error (attached).

Yeah, remember that some of the mingw-dlls will be in PATH if mingw is installed.

{l Code}: {l Select All Code}

Plugin_BSPSceneManager.dll
Plugin_OctreeSceneManager.dll
Plugin_OctreeZone.dll
Plugin_PCZSceneManager.dll

Neither of these are used, and I don't think we will end up using them either, so they are safe to remove. (I tested and removed those from the plugins.cfg files recently, I guess they have been left from an ogre example file and never been removed earlier.)

openal32.dll is only needed if OpenAL is not installed, so either have to ship it with the installer, or make the installer ask for downloading the openAL runtimes from creative. (Which I don't think is open source, so I'm not sure if we can't include the installer for that with the game.)

{l Code}: {l Select All Code}

CEGUIFalagardWRBase.dll
CEGUISILLYImageCodec.dll
SILLY.dll

I am quite sure I had to have these files to run OD earlier, so either stefan has them somewhere available, or they are not needed anymore after stefan improved the gui-related code. (Maybe the SILLY dll-files are only needed for specific image formats?)

plugins.cfg.linux and plugins.cfg.windows seems to be old leftovers, so they are safe to remove.

There are also some old pthread dlls in the dep folder that we probably don't need anymore. (Under the "old"-folder)

[StefanP.MUC]

svenskmand:
This error is from my latest commit. The dlls named "plugin_*" and The two OgreRenderer dlls (GL and D3D9) need to be in the "plugins" folder now.

Ah, ok, then we better keep the two from MingW (libstdc++-6.dll and libgcc_s_dw2-1.dll).

I'm not sure about CEGUI. I don't have it in my path or something (if I delete CEGUIBase.dll for example I get an error). When I did the GUI cleanup I removed a lot of unneeded code, includes and stuff (all we do with CEGUI now is calling a single bootstrap method and define thegui files to be loaded, more or less). The SILLY libary is a sub-project from CEGUI and seems to be be more or less dead (added to project 2006, only change to it 17 months ago).

edit: Ah, seems that SILLY is only used when CEGUI is not using any other image loader (we use Ogre, so CEGUI takes the image loader from Ogre). This means, we don't need the Silly dlls.
edit2: The Falagard dll is needed, sorry. I did a mistake there while testing...

edit3: updated the DLL list on the Wiki again with these infos.

[svenskmand]

What about the OgreRTShaderSystem.dll?

[oln]

It's needed.

[StefanP.MUC]

And it's not a ogre plugin, but an ogre basic dll (so it needs to be in the root folder, not in plugins).

[StefanP.MUC]

This is how it should look in the official release versions (no changes for the dev versions):

Old: "/models" (conatining *.mesh) and /materials (second with subfolders conatining pictures and .material and shader files). These are not needed in anymore.

New: new folder "graphics/" contains four .zip files:
- models.zip (containing all .mesh files)
- materials.zip (containing all .materials files, also the one from shader)
- textures.zip (containing all the texture picture files)
- shader.zip (containing the content of the RTShader folder)
All files in the zips must be in the zip root (Ogre doesn't support subfolders in zip files).

[svenskmand]

So I assume that the ressources.cfg should be change accordingly to use zips? If so how?

[StefanP.MUC]

Here is the whole file, I commented the parts that needs to be changed. The [Graphics] is the relvant one. So the "FileSystem" calls need to be replaced by "Zip" calls with the appropiate zip files. So the release package script needs to put all files from the folders into the zip files and put all teh zip files into the "ODroot/graphics" folder.

{l Code}: {l Select All Code}

[Graphics]
#this part from this:
#FileSystem=/materials/RTShaderLib
#FileSystem=/materials/RTShaderLib/materials
#FileSystem=/materials/scripts
#FileSystem=/materials/textures
#FileSystem=/models
#to this:
Zip=/graphics/materials.zip
Zip=/graphics/models.zip
Zip=/graphics/shader.zip
Zip=/graphics/textures.zip

[GUI]
FileSystem=/gui

[Music]
FileSystem=/music

[Sound]
FileSystem=/sounds


[svenskmand]

Any particular reason why the gui, music an sounds folders are not going to be zipped?