How important are libre development tools to you?

How important are libre development tools to you?

Postby nnesse » 07 Dec 2014, 05:08

I've been wondering how imporatant the use of free (as in freedom) development tools are to the average aspiring game developer here. I see many instances of projects that settle for a kind of theoretical support of free software tools but in practice a number of non-free programs are really driving the project forward. Inadiquite support of Blender is a good example. Many open source game engines import assets via COLLADA but Blender does not support exporting animation data within COLLADA (at least last time I checked). I would argue that a game that depends on animated models but is using COLLADA is making a questionable compromise. I think it's similarly problematic to use cmake to generate Visual Studio project files when MinGW is a viable option.

What I worry about is how these sorts of decisions can change the makeup of a project's contributers to the point that directions are taken that work against the free tools. There ought to be a network effect where every free software project elevates several others. So, for myself, I put the similarly strict requirements on the tools I use as the code I link with. Since so few libraries and tools put the same priority on this as I do I've had to do a lot more things myself from scratch. I've started working on a graphics/game engine with a tight integration with OpenGL, Blender, and various GNU software. It's been fun but time consuming. Since so many engine projects die out I've taken a highly modular approach to development to the point that the components can at least be used even if the project dies out (see https://github.com/nnesse). I would be interested in working with anyone who is interested in low level tech and willing to take the more difficult road of only building on software with clear libre pedigree's.
nnesse
 
Posts: 21
Joined: 02 Dec 2014, 00:39

Re: How important are libre development tools to you?

Postby oln » 07 Dec 2014, 15:21

Shouldn't you also use a website that uses free software then?
User avatar
oln
 
Posts: 1014
Joined: 26 Oct 2010, 22:16
Location: Norway

Re: How important are libre development tools to you?

Postby nnesse » 07 Dec 2014, 18:52

I assume you're talking about github vs gitorious? I've been pushing my code to both sites just to maximize the chance of getting some feedback. I agree that by my own arguments gitorious (or somthing comparable) ought to be used for collaboration. I just created a group on gitorious for this purpose: https://gitorious.org/+libre-gamedev-tech. I'm trying to move my current gitorious projects (gltext, glbindify) to the team page but the site keeps timing out on me. Just message me if you want to join the group

Rereading my original post I've realized that my definition of what software I would like to avoid is not clear enough. The tests that I apply is that all software I would use or be interested in helping to develop are:

1) Must be FOSS itself
2) Must provide superior support to other FOSS software and otherwise not encourage the use of non-free software
3) Must be buildable using software also following #1 and #2 (apply recursively)

That is, the whole software stack behind the engine and tools can be built from scratch and used effectively without leaving the FOSS world.
nnesse
 
Posts: 21
Joined: 02 Dec 2014, 00:39

Re: How important are libre development tools to you?

Postby onpon4 » 07 Dec 2014, 20:21

I only use libre software and encourage others to do the same, but it sounds like you're saying that stuff like 3-D models that were made with proprietary software should be rejected for this reason. Well, I would disagree with that conclusion. If reading it requires proprietary software, of course it should be rejected. But if it's just that someone else sacrificed their freedom to make it, then what's done is done, and there's really no point in rejecting it.

Similarly, I wouldn't reject a useful contribution to one of my programs if the contribution was typed in a proprietary text editor, regardless of whether or not the features of the proprietary editor made the work easier.
User avatar
onpon4
 
Posts: 382
Joined: 13 Mar 2014, 18:38

Re: How important are libre development tools to you?

Postby nnesse » 07 Dec 2014, 20:37

What I'm getting at is that the infrastructure that I depend on should not have a propritary bias. The original source of the contribution doesn't matter so long as it can be conditioned into a form where it fully ineroperates with the FOSS tech. Going back to the Blender case, If I accept a model with animations that cannot be imported into Blender then my project is now dependant on non-free modeler from a development process standpoint because only people who can afford and are willing to run such programs can improve or change that animation. The COLLADA format was supposed to provide the neccisary interoperability but in practice it rarely works as intended. Every tool has their own interpretation of the standard so when COLLADA is used you are effectively adopting a dependency on the program used to create the model. It's a similar story with cmake's ability to generate MinGW makefiles on windows. If user's are invited to generate Visual Studio solutions then the MinGW toolchain approach will only work intermittantly if ever. I understand how tempting making these kind of compromises is which is exactly why I would like to make some tech that improves the pure FOSS development experience.
nnesse
 
Posts: 21
Joined: 02 Dec 2014, 00:39

Re: How important are libre development tools to you?

Postby asdv » 11 Dec 2014, 21:00

For me a lot off creative FOSS software just lacks the usability I need to spend my free time using them. For example having recently had to try and compile something (SDLgfx) with both Visual Studio and MinGW I personally can’t see MinGW as a viable option. The amount of pain required to get everything set up and not-working was just too much.
And God help you if you want to compile for 64-bit. In Visual Studio it was as easy as choosing x64 on a drop-down list and I was able to find that option without consulting a manual or even googling since things are somewhat logically laid out.
Since you’re already on Windows I can’t see it mattering if you have to install VS.

And I don’t think I’ll ever be able to replace the adobe suite. Just a simple thing like I can’t find a single FOSS paint program that lets me do something as basic as move/copy multiple layers blows my mind.
GIMP is a perfect example; the fact that it took them over a decade of people complaining before they implemented a single window mode speaks volumes of what they think of their (potential) users. GIMP in general also lacks any discoverability whatsoever and as long as they spend years fiddling around with GEGL instead of writing a coherent design document I don’t see that changing.
I don’t want it to sound like I’m coming down too hard on FOSS, I use Linux most of the time myself, but the tool chain needs a lot of improvement.
asdv
 
Posts: 14
Joined: 30 Mar 2014, 15:55

Re: How important are libre development tools to you?

Postby nnesse » 13 Dec 2014, 06:43

I agree there are many projects that need a lot of work. That said the community should be looking for the best emerging solutions and rally around them. A lot of people have understandably bad memories of working with MinGW , buggy, terrible web presence, etc, but now we have MinGW-64 (http://sourceforge.net/projects/mingw-w64/) which I have personallly had good experiences with and can reccomend. In terms of getting things built with MinGW there are two projects worth looking at. MXE (http://mxe.cc/) which is a Linux makefile system which batch cross compiles a long list of popular software and libraries. If you want to work on windows take a look at the MSYS2 project (http://sourceforge.net/projects/msys2/) which is a Cygwin/MSYS replacment with good support and a real package manager which includes quite a few MinGW packages. Notibly, SDL2_gfx is built by MSYS2 and MXE so you'll probably find a solution to your build problem in one of those places.

There definitely are deficienies in the toolchain but there is also a social problem. There are some high profile projects that have a history of simply not accepting MinGW compatibility patches, the primary maintainers seem to only care about business use cases. What I want to do is get in touch with other developers who are interested in eliminating some of the biggest pain points and make new software for game developers that feels more like it belongs to the FOSS community. I can't really comment too much on the art side as I am mainly a programmer. My primary concern is the extent to which propritary tools have become institutionalized to a point where a free tool user cannot effectively contribute because there are no formats supported by both tools that preserve all the information. I don't know if the same kinds of issues exist between Photoshop and GIMP as there are between 3D modelers.

My next project, which is going to take awhile (and probably require collaborators) is a real-time renderer that operates as directly as possible on blender data. My approach will be to make a blender plugin that pickle's (http://www.picklingtools.com/) a subset of blenders RNA data then write a C++ library that loads the pickled data and creates corresponing GLSL shaders and provides an interface to control animations. The renderer will be a pluggable system mostly concerned with marshalling the data and doing the animation math. This way anyone who wants to can use the project to take different approaches to the actual rendering. Anyone who wants to follow the development or help can find it on gitorious: https://gitorious.org/blender-to-glsl
nnesse
 
Posts: 21
Joined: 02 Dec 2014, 00:39

Re: How important are libre development tools to you?

Postby gouessej » 11 Jan 2015, 22:31

Hi

nnesse {l Wrote}:My next project, which is going to take awhile (and probably require collaborators) is a real-time renderer that operates as directly as possible on blender data.

In my humble opinion, it's a bad idea as the Blender format isn't an interchange format, it's the native format of this modeler and nothing else. I think that it's the job of the modelers to provide a correct support of interchange formats. I like Blender but I have had the same troubles with animations and since its version 2.50, most importers for Quake formats have been dropped. When I succeed in updating the import scripts, the Collada export only gives me the very first frame :( Now I use the importers of my engine to manage those files and I export all key frames in a single OBJ file.

I understand your position. I try to stay far from proprietary softwares. The few native libraries I use are compiled without Visual Studio. I'd like to build OpenJDK without it too. I agree with you about the network effect. I often talk about the convergence of the struggles. We are stronger together. If all free softwares developers really wanted to eradicate proprietary softwares from their projects, it would increase the solidarity between them. Instead of that, I see lots of people who only care about their own things. For example, the homepage of the Neo900 shows a link to a Twitter account and a Youtube account whereas Identi.ca and GNU Mediagoblin are worth a try. I think that it's a question of priority and motivations. We shouldn't assume that all free softwares developers work on free softwares for ideological reasons.

Finally, I remind you that the preservation and the development of the alternatives are time consuming. Lots of people just want to use something convenient. I want to regain control on my data. I'm interested in self hosting and net neutrality too. I think that someone who isn't really convinced that free softwares are more "fair" would do lots of efforts to show that using only free softwares is a viable path. Actually, if you really want to be sure to stay away from proprietary softwares, you should put your own Git repositories onto your own server.

Edit.: I still have to go away from a few proprietary craps...
gouessej
 
Posts: 68
Joined: 13 Aug 2012, 18:28
Location: France

Re: How important are libre development tools to you?

Postby nnesse » 12 Jan 2015, 11:36

I have made some adjustments to my approach to my blender project that might address your concern. The goal was never to copy the blender format completely. I was just going to filter it, removing objects and properties that a real-time graphics engine would not need. After exporing this idea further I found that there was more data in the format that I didn't need than I did. Also many of the fields that were potentially useful were not adaquitely documented.

So now I'm going with creating a more conventional exporter but I still try to match the blend file's data structure so that tools and blender addons I create later can cater to techichal artists without fighting the format. The refactored project is here: https://gitorious.org/blender-asset-pipe/blender-to-lua. The output format is documented in the first part of the export script. Since blenders material system's has poor documentation and naturally has many properties centered around raytraced rendering I am only focusing on exporting animation and geometric data for now. At a later point I will either create a material editor that runs outside of blender or use the PyNodes API (used by cycles but currently undocumented) to create a node based material editor in blender tailored for real-time rendering.

I have some profressional experience working with people from the AAA gaming world and one of the things that I learned is that game studios usually avoid getting dependent on file formats like Collada, they will integrate as tightly as possible with their artists tools, even creating custom GUI's within them so that the tools will always produce content the engine can consume. I want to provide a solutions like that but tailored around the best available free tools instead.

I prefer self-hosting for the same reasons but in the early days of a project like this I want to stay focused on the work and not maintaining a VPS or home server. There is a tipping point where I would setup all the neccisary SCM and collaboration tools on my own server but I'm not there yet.
nnesse
 
Posts: 21
Joined: 02 Dec 2014, 00:39

Re: How important are libre development tools to you?

Postby nnesse » 24 Sep 2015, 08:15

I moved this work to notabug.org some time back since gitorious went down: https://notabug.org/nnesse/. I've also mirrored it at https://github.com/nnesse for extra visibility.

Since my last post I've improved blender-to-lua quite a bit. I also created a new project called b2l-tools that includes a material editor tool that is capable or previewing B2L files with animation playback and custom shaders. The material editor parses the GLSL of the user specified shaders and maps the inputs to widgets so you can experiment with the shading in real-time. It can also save out the shader parameters to file to make it easier to replicate the rendering in a separate game or engine. It's a very rough around the edges tool but perhaps someone here could still make use of it.

The next steps for me are going to be to broaden the scope of the material editor to include scene viewing/editing and isolate the rendering code to allow it to evolve into a proper graphics engine.
nnesse
 
Posts: 21
Joined: 02 Dec 2014, 00:39

Re: How important are libre development tools to you?

Postby Vandar » 24 Sep 2015, 10:27

For me it's not important if a tool is open source. Freeware is welcome if it does the job, also closed source tools from the shelf, which cost money.

Important is, that the benefit is greater than the price, and that they will stay with me for a while. E.g. I don't like rental models, where I have to pay a tool freshly each month or so.

On the other hand, my current tool set only contains open source tools, but that is just by chance.
I have project-ADHS, sorry.
User avatar
Vandar
 
Posts: 276
Joined: 17 Mar 2014, 14:32

Re: How important are libre development tools to you?

Postby nnesse » 24 Sep 2015, 18:44

There is a problem with that reasoning however. By the nature of their business proprietary software vendors not selling subscription software will tend to insert non-backwards compatible changes over time, making it amount to a subscription model either way.

There are also practical problems with depending on too many proprietary gamedev tools. It is often not possible to share data and algorithms between your code and your tools as freely as might be possible in an open source toolchain. There may be algorithms and data best applied in your game that the tool will only give you the option to "bake" to a much larger and less flexible data set. Likewise there may be transformations you are doing at run-time that would be best applied ahead of time in the tools.

A good example of this is a node based terrain generator I used one time. I got to a point where I realized what I wanted was for the terrains to be generated at run-time using seed data that changed based on what kind of challenge the user wanted. The tool neither provided the algorithms as a library or documented any of the formulas so the most I could do was to pre-generate a large set of possible maps which would have both used up lots of space and still not offered the same flexibility as my preferred approach. I don't think the lack of documentation of the formulas was an oversight. I suspect the developers presumed that having the math obscured in the compiled code was making it more difficult for others to create a competing tool.
nnesse
 
Posts: 21
Joined: 02 Dec 2014, 00:39

Re: How important are libre development tools to you?

Postby Vandar » 25 Sep 2015, 13:16

nnesse {l Wrote}:Likewise there may be transformations you are doing at run-time that would be best applied ahead of time in the tools.


In this case, you can also extract the transformation into a converter, which is run once in the release or setup process.

The other points might be valid, but they were no problems for me so far. And I must admit, even that I use open source tools, my will to change them is small. E.g. I'm using paint.net, it's my favorite graphics tool, and I really don't want to install a development environment for that. So in effect, it's the same for me as a closed source tool of equal capability. The reasons why I am using it, is that it costs nothing and it's small, easy to use and does what I need. Open source was not a prime reason for me to to chose paint.net over other tools.

If a tool doesn't do what I need, e.g. your landscape generator, I seldom try to change the tool, but I look for something that fits better.

Finally, I can understand that the landscape generator developers want to keep the formulas secret. If you want to make money with a product, you'd be stupid to publish such essential information - any competitor in the market would make use of it and you'll lose the advantage that you have.

That's different in the free/open-source world, but for companies which make money by producing and selling software, it's the right thing to do, I think.
I have project-ADHS, sorry.
User avatar
Vandar
 
Posts: 276
Joined: 17 Mar 2014, 14:32

Re: How important are libre development tools to you?

Postby nnesse » 25 Sep 2015, 18:17

Well in the case of the terrain generator it did do what I wanted, I just wanted to be able to replicate it at run time rather than export it. It's not so much about modifying the program as it is simply having visibility into it. Another example of this problem was with a contract I did awhile back where a long sequence of images had an effect applied to them. Without the effect the images compressed excellently so it would have been much better option to apply the effect in a shader and compress the originals. Regarding doing transformations ahead of time, yes you definitely can rely on external converters etc but in a complex project the fewer steps between a content creation and run-time the fewer mistakes that will be made and the faster people can work.

What I'm talking about I think applies more to 3D game development where the technical complexity tends to be higher. Medium to large players in this field will often have a support contract with all of their 3'rd party software they use so even if they don't have access to source they can always pay to get a bug fixed or a change made in the time frame they need. For an open source 3D project this is sort of a double blow: the cost of making the tools actually needed to be productive in open source may be too high, and any proprietary tools you do use can't be used as effectively as commercial players can.
nnesse
 
Posts: 21
Joined: 02 Dec 2014, 00:39

Re: How important are libre development tools to you?

Postby Vandar » 30 Sep 2015, 14:53

Well, there is Blender, and it has a impressive even if not always easy history. I never could wrap my head around Blender, but I see impressive things done with it. These days I ponder about installing Blender again, and try to learn it - third time, this would be. Actually I'd only need a mesh modeller with PovRay export, but it's difficult to find a good one.
I have project-ADHS, sorry.
User avatar
Vandar
 
Posts: 276
Joined: 17 Mar 2014, 14:32

Re: How important are libre development tools to you?

Postby Julius » 30 Sep 2015, 16:07

Uhm, when it comes to working on models propitiatory mesh formats like .max are a real headache, and sadly the exporter and importers for exchange formats like collada of fbx leave much to desire. In the end (except for the most basic mesh exchanges) forget about collaborating on a mesh if you aren't using the same software (and more or less even the same version of that software).
So for most purposes this leaves only Blender, at it is the only program that everyone has and that runs on everyones computer. Using .blend is also pretty much the only way you can store "source files" for your meshes in a way that someone else down the road even has a chance to ever continue where you left off; although sometimes that might mean that you have to go into the extensive version archive of Blender and download an older one. Luckily you can easily have multiple different versions installed on the same PC which makes "upgrading" a file usually possible.

The only other FOSS modeling tool that is worth a try (as it is much easier and quicker for organic modeling) is Wings3D. But I would suggest you set it to Blender3D's camera control from the beginning as at some point you will have to use Blender at least for some file converting, and being used to the controls prevents a lot of frustration.

@Vandar: Wings3D has a pov-ray exporter no?
“You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete” - Buckminster Fuller
User avatar
Julius
Community Moderator
 
Posts: 1942
Joined: 06 Dec 2009, 14:02

Re: How important are libre development tools to you?

Postby Vandar » 01 Oct 2015, 13:33

I didn't know about Wings3D. It looks very interesting to me, I'll try it later the day. Thanks for the suggestion!

Edit: Installed and gave it a quick try. Seems to be exactly what I need. Povray export is good, it even seems to have a povray integration for previews. So it seems to fit perfectly into my toolchain.

Thanks again! Saves me from the horrible Blender UI.
I have project-ADHS, sorry.
User avatar
Vandar
 
Posts: 276
Joined: 17 Mar 2014, 14:32

Who is online

Users browsing this forum: No registered users and 1 guest

cron