0.5 development: improved addon system

0.5 development: improved addon system

Postby odamite » 28 Jul 2012, 18:33

I'm starting this thread to discuss and announce progress regarding improved addon system planned for 0.5 release.

Things we need:
  • Easy way for users to upload levels and levelpacks
  • Simple and usable way to find and download other's addons
  • Make a secure system to prevent spam and hackers
  • Simple rating system to help find quality content
First I would like to share a mockup about in-game upload process. I wanted to keep it as simple as possible. I appreciate any feedback and suggestions. :)
upload.png
User avatar
odamite
 
Posts: 166
Joined: 16 Jan 2012, 16:28

Re: 0.5 development: improved addon system

Postby acme_pjz » 28 Jul 2012, 19:05

Hi odamite,

The process just looks like the uploading process of X-Moto. Manual moderation is needed, just like what X-Moto does. I have designed a level for X-Moto, and after uploading level I received suggestions from moderator and improved level and upload again, it repeated for some times before final publish. For reference you can look at this page http://xmoto.tuxfamily.org/index.php?page=sendyourlevel

PS: maybe level name and description can be loaded from level file (currently level name is present, maybe we can add description, author and license into it)
Some of my open source games on GitHub
User avatar
acme_pjz
 
Posts: 665
Joined: 10 Dec 2009, 15:32
Location: PeeKing, China

Re: 0.5 development: improved addon system

Postby Edward_Lii » 30 Jul 2012, 10:47

Hello,

odamite {l Wrote}:Simple rating system to help find quality content

I've been thinking about this recently and maybe we don't need a rating system.
Allowing ratings by users will require some form of database storage, although not impossible I would like to keep the server load as low as possible.

We could also go for the same approach as STK, certain addons can be featured, placing them at the top of the addon list.

acme_pjz {l Wrote}:PS: maybe level name and description can be loaded from level file (currently level name is present, maybe we can add description, author and license into it)

We could add a description and author field to the levels I guess.
So when the user uploads its level the fields will be auto filled but he can still change/extend the description.

About the license field, all (public) levels will have to be licensed under GNU GPLv3.
Although we might also allow the usage of CC0 so a license field would be needed.
From,
Edward_Lii
User avatar
Edward_Lii
MnMS Moderator
 
Posts: 777
Joined: 20 Dec 2010, 16:46

Re: 0.5 development: improved addon system

Postby acme_pjz » 30 Jul 2012, 11:00

Edward_Lii {l Wrote}:About the license field, all (public) levels will have to be licensed under GNU GPLv3.


I don't think so. There are even GPL games with proprietary levels, for example, Frogatto & Friends, Nikki and Robots, PlaneShift, etc. I think our levels (and uploaded levels) could also be CC-BY 3 or CC-BY-SA 3 or even proprietary (of course, not recommended ;) ), but it is up to the level author.
Some of my open source games on GitHub
User avatar
acme_pjz
 
Posts: 665
Joined: 10 Dec 2009, 15:32
Location: PeeKing, China

Re: 0.5 development: improved addon system

Postby Edward_Lii » 30 Jul 2012, 11:16

Hello acme_pjz,

acme_pjz {l Wrote}:I don't think so. There are even GPL games with proprietary levels, for example, Frogatto & Friends, Nikki and Robots, PlaneShift, etc. I think our levels (and uploaded levels) could also be CC-BY 3 or CC-BY-SA 3 or even proprietary (of course, not recommended ;) ), but it is up to the level author.

I agree that the author can choose any license he likes, but there's a small problem with the creative commons licenses.
It's a bit of a grey area, my intial thought was to license the levels distributed with the game CC-BY(-SA), but after reading this:
http://wiki.creativecommons.org/GPL_compatibility_use_cases
we decided to settle on public domain (CC0), just to be sure. ;)
From,
Edward_Lii
User avatar
Edward_Lii
MnMS Moderator
 
Posts: 777
Joined: 20 Dec 2010, 16:46

Re: 0.5 development: improved addon system

Postby ctdabomb » 30 Jul 2012, 15:08

Edward_Lii {l Wrote}:
odamite {l Wrote}:Simple rating system to help find quality content

I've been thinking about this recently and maybe we don't need a rating system.
Allowing ratings by users will require some form of database storage, although not impossible I would like to keep the server load as low as possible.

We could also go for the same approach as STK, certain addons can be featured, placing them at the top of the addon list.

yes. I like this better than the rating thing(though STK also does have ratings as well as featuring.)
Edward_Lii {l Wrote}:we decided to settle on public domain (CC0), just to be sure. ;)

good choice. my first thought though was if people uploaded the same levels ass other people did. but that is what moderation is for.
Some people are like slinkies... not really good for anything, but you still can't help smiling when you shove them down the stairs.
ctdabomb
 
Posts: 1075
Joined: 13 Dec 2011, 21:21
Location: halfway there

Re: 0.5 development: improved addon system

Postby leovilok » 02 Aug 2012, 23:01

I think that for the themes at list all the licencies that can be found on OGA should be accepted. That includes CC-BY and CC-BY-SA.
So these licencies should be allowed for level(pack)s too. But it should be written that with these licencies it will never be able to go in the main package.
leovilok
 
Posts: 32
Joined: 22 Oct 2011, 20:22

Re: 0.5 development: improved addon system

Postby Edward_Lii » 11 Sep 2012, 15:41

Hello all,

I know it's been a long time. I've been extremely busy and didn't have any time to work on Me and My Shadow.
I'm still quite busy but at least I have got some time on my hands now.

Anyway, back on topic. The server side is almost done.
We can divide the upload system in three parts: uploading, moderating, downloading.

Uploading
This part will mostly be server side, the client only has to send the HTTP post request.
I try to keep the upload script on the server as simple as possible.
The only measure to prevent spamming at the moment is a maximum size for the moderation queue.
  • Check if the queue isn't full.
  • Validate the (user) input.
  • Create queue file.
  • Write header to the queue file.
  • Write body to the queue file.

The layout of the queue file is pretty simple:
type = <TYPE>
name = <name>
description = "<description>" (encoded with json_encode( ... ) )
update = <1 or 0>
username = <username>
email = <email> (not verified)
-- (delimiter)
<BODY>



Moderating
The moderation should be easy and straightforward.
I plan on making a few bash scripts that can handle each step, a GUI can always be made later.

The work flow should consist of:
- Receive the queue.
- Check a queue entry (header and body).
- Adjust entry if needed.
- Commit entry to the addon git repository.
- Remove the entry from the queue.

I think the biggest problem is going to be the removing of the queue entry on the server side.
We don't want anybody to be able to remove it, so there needs to be some form of authentication for the moderators.
I think a simple username,passwordhash file will do the job.

Concurrency might be a problem, but I think adding a check to see if the local working copy of the git repository is up to date before commit will do the job.


Downloading
This system is already present in the game but needs to be extended as described here:
http://forum.freegamedev.net/viewtopic.php?f=48&t=2959&p=29171&hilit=addon#p29138

Also part of the downloading is the levelpack management.
If an user uploads its levelpack and it gets pushed to the addon repository, he will most likely test it out.
This means that he will have two identical levelpacks installed, one in the addon path, the other in the custom path.
At the moment the game doesn't handle this too well (not at all. :p )

Feedback is welcome! ;)
From,
Edward_Lii
User avatar
Edward_Lii
MnMS Moderator
 
Posts: 777
Joined: 20 Dec 2010, 16:46

Re: 0.5 development: improved addon system

Postby odamite » 25 Sep 2012, 13:10

Hello Edward_Lii,

I've been pretty busy lately and I think that I'll have some time to work on Me and My Shadow soon.

But anyway, the system seems great to me. In my opinion it's good to keep it quite simple but still functional. :)

The server side looks good and working. However I think that there should be more advanced way to prevent spamming. If I understand correctly, at the moment spamming the queue to maximum will prevent others completely. We don't want that... :think:

I would like to know what protocol the moderation part is going to use? Is it based on PHP and HTTP? I think that the best would be FTP bacause doesn't it provide secure authentication out of the box? However I'm not sure what kind of things sourceforge allows...

Also we should think about implementing a rating system. This isn't obligatory but just nice thing to have. This could be created using a simple text file and HTTP POST and it shouldn't require any "spam" prevention system.

Yeah, we already have pretty good plans and something is already implemented. It's going to be great to see the final system and people using it. :cool:
User avatar
odamite
 
Posts: 166
Joined: 16 Jan 2012, 16:28

Re: 0.5 development: improved addon system

Postby Edward_Lii » 25 Sep 2012, 13:41

Hello odamite,

odamite {l Wrote}:The server side looks good and working. However I think that there should be more advanced way to prevent spamming. If I understand correctly, at the moment spamming the queue to maximum will prevent others completely. We don't want that... :think:

I understand what you mean, hopefully we won't need a more advance spam prevention system, though.
The biggest problem is excluding/blocking the spammer, since there's no form of authentication we can only block IP addresses.
But without too much effort the spammer can easily use a proxy and considering that the game has in-game support for using a proxy... :|

odamite {l Wrote}:I would like to know what protocol the moderation part is going to use? Is it based on PHP and HTTP? I think that the best would be FTP bacause doesn't it provide secure authentication out of the box? However I'm not sure what kind of things sourceforge allows...

At the moment I use a text file (only readable by the scripts) that contains the usernames and the hashes of the corresponding passwords.
AFAIK Sf doesn't provide https for the project web so that is a bit insecure..., but the only action that requires authentication is the deletion of the (server side) queue files.
Basically everyone can retrieve the list of queue entries from the server and download each queue file without any authentication.

The rest of the moderation happens locally except for the uploading/committing of the (moderated) queue entry to the git repository.
That requires the Sf username and password.

{l Code}: {l Select All Code}
User uploads level(pack) --> put in the queue --> moderator downloads (one) queue entry --> moderator changes/corrects/tests queue entry --> moderator commits to git repository (Sf authentication) --> moderator removes queue file from server (own authentication).

The last two steps are interchangeable, the moderator can even remove a queue file from the server without even looking at it.
One downside of the moderation happening locally is that in the time it takes the moderator to test and commit the queue entry another moderator might have done the same queue entry.
I will probably force the local git repository to be up to date before committing so a moderator should see that someone has beaten him to it before committing, but the duplicate effort has already been done.

odamite {l Wrote}:Also we should think about implementing a rating system. This isn't obligatory but just nice thing to have. This could be created using a simple text file and HTTP POST and it shouldn't require any "spam" prevention system.

This can be done quite easily, but there's one problem.
The addon list is retrieved from the git repository so if we add the ratings to that file it won't (auto-)update for each rating.
Which on the other hand might be a good idea since one can observe the delta of the rating of addons before applying them, making it easy to spot any "fraud".

odamite {l Wrote}:Yeah, we already have pretty good plans and something is already implemented. It's going to be great to see the final system and people using it. :cool:

It still requires a lot of work and testing, but we're getting somewhere. ;)
From,
Edward_Lii
User avatar
Edward_Lii
MnMS Moderator
 
Posts: 777
Joined: 20 Dec 2010, 16:46

Who is online

Users browsing this forum: No registered users and 1 guest

cron