[Meta] FreeGameDev changes NEW: https://freegamedev.net

Re: [Meta] Proposed FreeGameDev changes

Postby Lopy » 21 May 2021, 20:11

Forum posts to appear on Mastodon feeds, that's bad, for me that is. That indicates we do not just disagree on means, but that our objectives are likely incompatible. Given that you are the last remaining admin, and that only 5 other people posted on this thread, I'm guessing I'll lose.

If the phpBB developers came over, made and maintained a Mastodon bridge for us overnight, with every other technical feature remaining the same, this would still drastically alter the nature of this forum and break certain usecases. Mastodon has been built as a social media platform, as such, it incentivises form of messages that would traditionally be seen on one. The same goes for Lemmy and it's tree-ed chat. This forum, on the other hand, caters to longer messages, linear discussions, etc. Now, at first current users might continue as usual, with only light interference from people coming over from other sites. However, more and more often someone answering through Mastodon would apply the standards from there even if answering to a topic originated from here, slowly making the behaviour of people more homogeneous. This is why keeping conversations on multiple sites with different visual identities is a good thing: it allows users to compartmentalise their behaviour easier. For instance, I wouldn't want all my news sources to be aggregated, as this would make it harder for me to analyse on first sight the credibility of the article. If I read an article about potted plants on fsf.org or potted-plant-consortium.example.org, I interpret it very differently.

Another issue with merging communities like that, is that even if you might end up with a group bigger than the biggest starting one, it comes at the cost of creating many very small/unary groups. You might manage to keep the initial splinter small, but if the new giant group does not cater well to someone's usecase, that person will have a suboptimal experience, or hopefully realise they where having one and exit.
Then, people in very small groups will have a much harder time gathering others to recreate something that corresponds to their specific usecase, because most people will either be in the giant group, or not see the point of not joining it instead. For instance, say you managed to agglomerate most of the free FPS community into a single game/federation of compatible games. This would single handedly solve the issues about FPSs we where discussing in another thread. However, this would also means standardising certain things. I for one like FPSs with exceptionally long TTKs. With that giant group in existence, finding enough players to play like that seems even harder than right now.
Lopy
 
Posts: 12
Joined: 19 May 2021, 00:21

Re: [Meta] Proposed FreeGameDev changes

Postby Julius » 21 May 2021, 22:29

Hmm, I think I get your point, but I find it rather theoretical and not really how these things in my experience develop on the Fediverse. Sure the occasional external contributor will pass by, but given how dead this community is, I see that as a net positive. But for the most part people that subscribe to a community on Lemmy (regardless from where) stick to the stated rules of that community and the moderators can also enforce that. In that sense (UI and feature wise) Lemmy is not quite comparable to Mastodon itself, which replicates many of the UI mistakes of Twitter.

Also I don't quite agree with your FPS example conclusions. Such an FPS would work like Quake/Xonotic, e.g. on connection to the server it downloads the local game-play rules & assets and thus you can play exactly the type of FPS you want and prefer. But because the overall "FPS federation" is much larger and less fragmented it is much more likely that there will be other people joining that server and trying out the game-play as you like it.
User avatar
Julius
Community Moderator
 
Posts: 3297
Joined: 06 Dec 2009, 14:02

Re: [Meta] Proposed FreeGameDev changes

Postby PeterX » 22 May 2021, 18:03

I (and I must emphasize that's only my personal view) think contrary to Lopy, in all points.

FGD.net has (afaik) not the wish to take-over all communication related to libre games on the internet. Bundling communication a bit would be cool, not bad, but it is not about a totalitarian approach if I understand it correctly.

Lemmy seems quite cool to me.

Lopy's favorized form of a forum wouldn't hurt either. Unfortunately there are not many libre game devs hanging around here. So if Lopy doesn't run his own site/channel nobody is going to do so. I personally feel quite busy developing my game, so I am not much of a help. But maybe Lopy has time and fun doing it?

Greeting
Peter
User avatar
PeterX
 
Posts: 270
Joined: 01 Oct 2020, 21:44

Re: [Meta] Proposed FreeGameDev changes

Postby drummyfish » 22 May 2021, 19:25

As @PeterX I am also for Lemmy now -- this is also just my personal opinion so take it as one data point for your decisions. I've used Lemmy for a while and it was very comfy. I think the tree structured conversations are much more readable and inviting to new members, while they still allow and even encourage long and insightful replies -- they just don't discourage short and cheap replies, but thanks to the structured nature it's also easy to filter these out visually (just take a look at reddit or hacker news, I think the discussions mostly result in quality additions to the post). Most comment sections on the Internet have evolved to be at least partially tree structured. I don't even really care about federation but I guess it might help get more members from other Lemmy instances into out community, so it might be good. Anyway I'd like to see both Lemmy and this forum up at least for a while, letting users decide where they want to settle, more freedom and choice is always better IMHO.
socialist anarcho-pacifist
Abolish all IP laws. Use CC0. Let's write less retarded software.
http://www.tastyfish.cz
User avatar
drummyfish
 
Posts: 448
Joined: 29 Jul 2018, 20:30
Location: Moravia

Re: [Meta] Proposed FreeGameDev changes

Postby Julius » 22 May 2021, 20:44

PeterX {l Wrote}:FGD.net has (afaik) not the wish to take-over all communication related to libre games on the internet. Bundling communication a bit would be cool, not bad, but it is not about a totalitarian approach if I understand it correctly.


I understood him in the way that he fears the typical "Mastodon culture" will take over this place and not the other way around. I agree that there is a certain risk to that, but I am not overly worried that there will be a clash of cultures in which one ceases to exist.
User avatar
Julius
Community Moderator
 
Posts: 3297
Joined: 06 Dec 2009, 14:02

Re: [Meta] Proposed FreeGameDev changes

Postby Julius » 29 May 2021, 22:08

Small update on this:

Regarding Lemmy... I still like it, but I by now I learned how lacking it is in moderation tools, which is kind of a no-go for us here with the amount of f2p games and gambling advertisement spam we get. I also don't make much progress with the server side of things necessary to host it, so I am starting to wonder if it is a good idea after all.
I also thought about Lopy's comments again, and while I still don't agree on the "culture" part, I kind of see how federation would not add much to a low traffic forum like ours.

So I guess we are back at Flarum? They recently released version 1.0 and it still looks like a nice forum replacement to me :-/
User avatar
Julius
Community Moderator
 
Posts: 3297
Joined: 06 Dec 2009, 14:02

Re: [Meta] Proposed FreeGameDev changes

Postby Lopy » 31 May 2021, 02:28

Flarum looks alright. I could not spot any missing feature, except maybe an absence of viewable account details, although that could be because I am not logged in there, or the info fields where empty.
The criteria I had about what the software make you write seem met.

Can it be used fully freely? The various instances I found all had proprietary JavaScript, however none was on all instances, which makes me think they might all be optional in the configuration. How free can you get it to be? I know the current forum has proprietary analytics, but it can be disabled client-side without any issue.
Lopy
 
Posts: 12
Joined: 19 May 2021, 00:21

Re: [Meta] Proposed FreeGameDev changes

Postby Julius » 31 May 2021, 02:44

It used to have google font integration, but I think that was removed. So AFAIK the core Flarum is fully libre even if it heavily depends on Javascript to function. But I will double check that.

There are loads of extensions though that are not fully libre but that would need to be seen on a case by case basis.
User avatar
Julius
Community Moderator
 
Posts: 3297
Joined: 06 Dec 2009, 14:02

Re: [Meta] Proposed FreeGameDev changes

Postby Julius » 05 Jun 2021, 00:08

I got fed up and made a Flarum again, find it on https://freegamedev.net

Seems to work nicely and the code-base seems much improved since I last tried it.

The "Projects" section is still somewhat under debate. Not sure what to do with it exactly. I think it might be better to keep it a bit more generic instead of having all these abandoned project sub-forums like here.

If it is generally liked and runs stable, we will probably lock the "Community" part of this forum and keep it as an archive only. For the hosted project forums we will have to see what to do exactly.
User avatar
Julius
Community Moderator
 
Posts: 3297
Joined: 06 Dec 2009, 14:02

Re: [Meta] Proposed FreeGameDev changes

Postby bzt » 05 Jun 2021, 08:31

Julius {l Wrote}:I got fed up and made a Flarum again, find it on https://freegamedev.net

Seems to work nicely
It does not work at all for me. Some js is blocked due to cross-site scripting and then the whole site collapses. I can see nothing essentially.
Could you make it without js or at least without cross-site scripting? (Just because it involves cdn doesn't make it safe)

I like it very much about phpBB that it is self-contained (no cdns) and works without js perfectly.

Cheers,
bzt
User avatar
bzt
 
Posts: 332
Joined: 23 May 2021, 21:46

Re: [Meta] Proposed FreeGameDev changes

Postby Julius » 05 Jun 2021, 10:11

That must be partially an issue on your side (what browser are you using?), as I spend a lot of time yesterday trying to eliminate exactly that, some CDN loaded emojis :( Totally without JS is not possible with Flarum though, and I have given up trying to find an option that can satisfy this requirement without reasonable trade-offs in other areas.

However I now noticed that I missed one "flatpickr.min.css" that is still loaded from a CDN. As that is just a CSS file it should not cause your issues, but I will try to eliminate it never the less. Thanks for the feedback.

Edit: the last external resource mentioned above seems to have been part of the "polls" plugin. I disabled it and filed a bug report. AFAIK it now should not load any resources from outside freegamedev.net and all javascript (and everything else) is fully FOSS.
User avatar
Julius
Community Moderator
 
Posts: 3297
Joined: 06 Dec 2009, 14:02

Re: [Meta] Proposed FreeGameDev changes

Postby trevor » 05 Jun 2021, 16:14

Julius {l Wrote}:I got fed up and made a Flarum again, find it on https://freegamedev.net


OH MY GOD!!! Julius, You killing forum. Please don't do it. I really love this place, where people talking about FOSS games. This phpbb forum is good. New forum is f**ing hipster-looking shit.
I'm sorry. My English is very bad.
trevor
 
Posts: 26
Joined: 27 May 2018, 11:21

Re: [Meta] Proposed FreeGameDev changes

Postby PeterX » 05 Jun 2021, 16:59

trevor {l Wrote}:
Julius {l Wrote}:I got fed up and made a Flarum again, find it on https://freegamedev.net


OH MY GOD!!! Julius, You killing forum. Please don't do it. I really love this place, where people talking about FOSS games. This phpbb forum is good. New forum is f**ing hipster-looking shit.

Ha, that's a matter of taste. Yes, the new one's look is a bit slick. But I like the new one. And a bit of experimenting doesn't hurt imho.
User avatar
PeterX
 
Posts: 270
Joined: 01 Oct 2020, 21:44

Re: [Meta] Proposed FreeGameDev changes

Postby Julius » 05 Jun 2021, 17:49

trevor {l Wrote}:
Julius {l Wrote}:I got fed up and made a Flarum again, find it on https://freegamedev.net


OH MY GOD!!! Julius, You killing forum. Please don't do it. I really love this place, where people talking about FOSS games. This phpbb forum is good. New forum is f**ing hipster-looking shit.


Sorry, I know this will not be in everyone's favor or liking :(

But we have been going back and forth on the topic what to do with the forums for like at least 3-4 years now, and there just isn't a solution that can satisfy everyone and fit within the limited hosting requirements that we have. And just continuing with these old forum is IMHO not sustainable either.

I wish I could think of a better option, but Flarum at least ticks 2 out of the 3 major complaints / issues we face.
User avatar
Julius
Community Moderator
 
Posts: 3297
Joined: 06 Dec 2009, 14:02

Re: [Meta] Proposed FreeGameDev changes

Postby bzt » 05 Jun 2021, 17:55

First thing first, what's the problem with phpBB? It's not looking great, I admit, but it works perfectly even with your server requirements. Why not put some css to it to make it look better?

Julius {l Wrote}:That must be partially an issue on your side
Nope. I have a very secure configuration, that's why I catch these.

Julius {l Wrote}:I spend a lot of time yesterday trying to eliminate exactly that, some CDN loaded emojis :(
Well, there's still a lot to do, it is full of jsdeliver.net urls.

Julius {l Wrote}:Totally without JS is not possible with Flarum though, and I have given up trying to find an option that can satisfy this requirement without reasonable trade-offs in other areas.
I would say then this is a shortcoming of Flarum. There's absolutely nothing on a forum that would require js, as phpBB demonstrates.
That being said, it's better if all js are hosted and served from the server, not from some shady third-party.

Julius {l Wrote}:However I now noticed that I missed one "flatpickr.min.css" that is still loaded from a CDN. As that is just a CSS file it should not cause your issues, but I will try to eliminate it never the less. Thanks for the feedback.
You're welcome, and thanks for checking it out! Yes, there's css too.

Julius {l Wrote}:Edit: the last external resource mentioned above seems to have been part of the "polls" plugin. I disabled it and filed a bug report. AFAIK it now should not load any resources from outside freegamedev.net and all javascript (and everything else) is fully FOSS.
I can confirm that all cdn urls have been removed. Still don't like it though, I have to enable js which always make me nervous.

Cheers,
bzt
User avatar
bzt
 
Posts: 332
Joined: 23 May 2021, 21:46

Re: [Meta] Proposed FreeGameDev changes

Postby Julius » 05 Jun 2021, 18:25

Thanks for confirming that I caught all the external stuff :) I agree with you that strictly speaking a forum shouldn't require Javascript, but there just isn't a really good option available that has a fully working nojs fallback :(

For reasons outlined elsewhere this specific phpBB runs on a really old version that is full of issues, but upgrading just doesn't work both for organizational and also technical reasons (every-time the current server admins touch even the slightest thing it breaks down totally). A more modern phpBB might be alright, but when starting completely new (for legal reasons we can't just transfer the personal data without user-consent, but IANAL), there are better options like Flarum in my personal opinion.
User avatar
Julius
Community Moderator
 
Posts: 3297
Joined: 06 Dec 2009, 14:02

Re: [Meta] Proposed FreeGameDev changes

Postby bzt » 06 Jun 2021, 01:45

Julius {l Wrote}:A more modern phpBB might be alright, but when starting completely new (for legal reasons we can't just transfer the personal data without user-consent, but IANAL), there are better options like Flarum in my personal opinion.
You have contradicted yourself here. You have said yourself why Flarum is actually a worse option.

Using an up-to-date phpBB has several advantages over Flarum:
- no javascript
- no migration nor wiping needed
- you can keep the old data and all the links (you just need to issue a few sql schema queries but keep the records and ids intact)

Have you considered how many links there are to this forum? Flarerpg forum redirects here for one. All those links will be broken with Flarum. With a new version phpBB those cloud be the same. I can help you with the sql if you'd like, there's no need to start a completely new forum.

Cheers,
bzt
User avatar
bzt
 
Posts: 332
Joined: 23 May 2021, 21:46

Re: [Meta] Proposed FreeGameDev changes

Postby Julius » 06 Jun 2021, 02:27

No you misunderstood :)

I like libre Javascript or rather see nothing wrong with it, but we had this discussion in depth before so no need to start it here again please.
But despite this, I think a forum should have a functional non-javascript fallback, which Flarum sadly hasn't (AFAIK you can only read and it looks bad).

As for the other points, again discussed a lot before, but even a new phpBB would require a complete restart as for both technical and legal reasons we can't just plumb the old database into a new system and upgrading the existing one simply doesn't work (it just breaks horribly). It's not about "help" that is required, it simply doesn't work technically (& in regards to administrative rights to the server) and we can't get user consent for transferring all the personal data either.

And last but not least, we do not plan to remove this forum here, so links will stay intact (as least if technically possible). The idea is simple to retire it and use as an read-only archive. However given the sorry state this forum/database is in, even after turning it read only it might break again, so I am thinking about turning it into a set of static pages. However I am not quite sure how that can be done best and if that doesn't change the link structure to be incompatible with external links.
User avatar
Julius
Community Moderator
 
Posts: 3297
Joined: 06 Dec 2009, 14:02

Re: [Meta] Proposed FreeGameDev changes

Postby bzt » 06 Jun 2021, 10:27

Julius {l Wrote}:No you misunderstood :)

I like libre Javascript or rather see nothing wrong with it, but we had this discussion in depth before so no need to start it here again please.
But despite this, I think a forum should have a functional non-javascript fallback, which Flarum sadly hasn't (AFAIK you can only read and it looks bad).
Yeah, you can say "it looks bad" without js, but I personally think it is a lot worse than that. You can only see the tags, that's all.

Julius {l Wrote}:As for the other points, again discussed a lot before, but even a new phpBB would require a complete restart as for both technical and legal reasons we can't just plumb the old database into a new system and upgrading the existing one simply doesn't work (it just breaks horribly). It's not about "help" that is required, it simply doesn't work technically (& in regards to administrative rights to the server) and we can't get user consent for transferring all the personal data either.
It does work. There's absolutely no technical issues, I've done it many many times, I could do it for you without a transfer. Even if the new database contains only the topics (with their ids and therefore keeping the links) is a much better solution that starting anew with Flarum. You don't know me so it is totally understandable that you won't trust the admin access to me, otherwise I could do this easily. In lack of admin access, here's how this could work:
1. install a brand new latest phpBB
2. send me the schema for both the current and the new database (not the data, just the schema. If no customization applied, then it's enough if you send me the exact phpBB versions)
3. I'll send you back a bunch of queries
4. create a backup of the database (this is very important, because I'll working blind here so I can give you no guarantee that those queries will work at first without hiccup)
5. you execute the queries I send
6. point the new phpBB to the updated database
7. if there's a problem, restore the database from backup, send me the exact error messages and got to step 3.
8. ta-dam!

But even if you insist to restart everything, if I were you I'd consider using the latest phpBB with migrating only the topics without the posts and users (so that 99% of the existing links would remain the same, for example this topic would still be accessible by the link "https://freegamedev.net/viewtopic.php?f=20&t=17001" just without the posts). Migrating only the topics is easy, works out-of-the-box and requires no user consent either.

Julius {l Wrote}:And last but not least, we do not plan to remove this forum here, so links will stay intact (as least if technically possible). The idea is simple to retire it and use as an read-only archive. However given the sorry state this forum/database is in, even after turning it read only it might break again, so I am thinking about turning it into a set of static pages. However I am not quite sure how that can be done best and if that doesn't change the link structure to be incompatible with external links.
You create a script (could be bash, python or php) that downloads each page with a webcrawler (wget or curl can do this) with a generated name (say 000001.html, you could match the extension but not required) and at the same time saves a rewriterule to a file with the url used and the assigned local filename. When you've finished, you upload the static files and the rewriterule list to an Apache.

Alternatively configure the server (filesystem and Apache as well) to accept filenames with "?", "&" and "%" in them and then you could use the webcrawler's output as-is. For example the command
{l Code}: {l Select All Code}
wget -B https://forum.freegamedev.net -r https://forum.freegamedev.net
will download the entire site into the directory "forum.freegamedev.net", use that directory as your new document root in Apache. The only problem with this approach is, filenames will contain special characters that not all file systems and Apache configuration like (that's what creating the rewriterule list solve).

Cheers,
bzt
User avatar
bzt
 
Posts: 332
Joined: 23 May 2021, 21:46

Re: [Meta] Proposed FreeGameDev changes

Postby Julius » 06 Jun 2021, 12:05

Thanks, the offer for help is appreciated. I understand that *technically* of course any database is salvageable as long as the data itself is not on a corrupted hard-drive or such. But as you have more or less outlined above, even if there are no unforeseen issues it is a lot of effort, for what at least IMHO is a worse outcome than just keeping an archive and start new with a much more user-friendly (at least of the far majority of the users) forum. This community is very much dying, and to a large part because the technical foundation is stuck in the early 2000, just as phpBB in general.

But in the end this isn't even up to me, I am only trying to do damage control, as I only have access to the DNS nameserver and zero access to the server these old forums run on. Charlie, the original founder of this project (and owner of the domain) is only around from time to time and is not very invested in the administrative side of things, and the two admins of the server (Hagish and Ghoulsblade) are still friendly to the cause but have not been actively involved in this community for years. Thus IMHO non of the parties involved are willing to spend the time and nerves go though a large forum database salvage operation even if you would help us with the hard parts (all the legal implications aside due to new GDPR legislation).

As for the static pages archiving, yes that sound about the same as what I googled so far, although there seems to be a tool to maybe automate it: https://www.httrack.com/
User avatar
Julius
Community Moderator
 
Posts: 3297
Joined: 06 Dec 2009, 14:02

Re: [Meta] Proposed FreeGameDev changes

Postby bzt » 06 Jun 2021, 15:21

Julius {l Wrote}:Thanks, the offer for help is appreciated.
You're welcome! Let me know if you want to give it a try.

Julius {l Wrote}:This community is very much dying, and to a large part because the technical foundation is stuck in the early 2000, just as phpBB in general.
I don't know the state of this community (I'm quite new here), but I know old technology isn't necessarily the cause. I mean, take a look at IRC, it's even older than phpBB, yet it supports huge communities.

Anyway, if you're decided to use Flarum, could you please install this add-on (or any other similar). Adding support for feeds would allow me (and others) to check the new posts without enabling js and turning off L7 firewalls. That would be awesome! (TBH I don't know this add-on, never used it, but looks promising. If you have any trouble with it at all, then I can create you a dependency-free, suckless rss.php to do exactly the same from Flarum's data.)

Cheers,
bzt
User avatar
bzt
 
Posts: 332
Joined: 23 May 2021, 21:46

Re: [Meta] Proposed FreeGameDev changes

Postby Julius » 06 Jun 2021, 15:44

Yes, I hope to add RSS feeds to it ASAP, but the problem is that (contrary to what I thought because I previously used the plugin you linked) there is currently no good RSS feed addon that is compatible with the latest version of Flarum :(

Someone on the official Flarum forum just posted a new one, but it seems to use a not very good way of generating the feeds so I am not sure if that would be a good way to implement it. See: https://discuss.flarum.org/d/27569-rss-feed
User avatar
Julius
Community Moderator
 
Posts: 3297
Joined: 06 Dec 2009, 14:02

Re: [Meta] Proposed FreeGameDev changes

Postby bzt » 06 Jun 2021, 20:02

Julius {l Wrote}:Yes, I hope to add RSS feeds to it ASAP, but the problem is that (contrary to what I thought because I previously used the plugin you linked) there is currently no good RSS feed addon that is compatible with the latest version of Flarum :(
Oh, that's a pity. As promised, here's my suckless rss generator that doesn't care about the overcomplicated, way-to-many OOP abstraction forced Flarum source at all. (I understand that it is written by amateurs for free, but calling it "delightfully simple" made my day :-) Just ROTFL :-D ) Digging into the code made me remember why I have left webdevelopment altogether years ago... Oh, and by the way, taking a look at the sql schema revealed that Flarum is storing the user's IP address with every post without the user's consent which is in a direct violation of GDPR. You should do something about it, turn this feature off if possible.

Julius {l Wrote}:Someone on the official Flarum forum just posted a new one, but it seems to use a not very good way of generating the feeds so I am not sure if that would be a good way to implement it. See: https://discuss.flarum.org/d/27569-rss-feed
Holy ****, you're right, it's downloading the page and parsing the html then downloads each and every discussion page as well...

Okay, here's my ca. 50 SLoC version. Note that I have no running Flarum instance, so I've used this and this as a resource. This means it should work, but haven't been tested. If you provide feedback, I'll fix it. One thing I couldn't figure out, is if the discussions table's slug field contains the sluggified title only, or the entire url part with the discussion id as well. Depending on this, there might be no need for printing $row["id"] in the guid field. Other minor thing, it's been a long time I've used php, so I'm not sure if htmlspecialchars needed with CDATA. Maybe not.

Otherwise I've optimized the sql query and did everything to keep it simple and resource friendly and independent to the current Flarum version. It should return at most 100 discussions with the most recent posts in descending order, displaying the discussion's title, the poster's username and relative time of the most recent posts.
{l Code}: {l Select All Code}
<?php
/* ------------------ set this to the Flarum config file -------------------- */
$config = @include("path/to/flarum/config.php");
$limit = 100; /* return that many records at most */
/* -------------------------- configuration ends ---------------------------- */

if(!is_array($config) || !is_array($config['database']))
    die("Configuration not found.\n");

/***** connect to the database using the configuration in Flarum's config *****/
try {
    $pdo = new \PDO(
        $config['database']['driver'].":host=".$config['database']['host'].
        ";dbname=".$config['database']['database'].
        (!empty($config['database']['charset']) ?
            ";charset=".$config['database']['charset'] : ""),
        $config['database']['username'],
        @$config['database']['password'],
        [
            \PDO::ATTR_ERRMODE => \PDO::ERRMODE_EXCEPTION,
            \PDO::ATTR_EMULATE_PREPARES => 0
        ]);
} catch(\PDOException $e) {
    die("Unable to connect to database: ".$e->getMessage()."\n");
}
if(!empty($config['database']['charset']))
    $pdo->exec("set names ".escapeQueryChars($config['database']['charset']).";");

/********************** get some data from the database ***********************/
try {
    $sql = "SELECT a.id, a.title, a.slug, a.last_posted_at, b.username, ".
        " TIMESTAMPDIFF(SECOND, a.last_posted_at, NOW()) as t, ".
        " DATE_FORMAT(a.last_posted_at, '%a, %d %b %Y %T') as d FROM ".
        $config["database"]["prefix"]."discussions as a LEFT JOIN ".
        $config["database"]["prefix"]."users as b ON a.last_posted_user_id = b.id ".
        "WHERE b.id IS NOT NULL AND a.is_private != 1 ".
        "ORDER BY a.last_posted_at DESC LIMIT ".$limit.";";
    $res = $pdo->query($sql);
    $rows = $res->fetchAll(\PDO::FETCH_ASSOC);
    $pdo->close();
} catch(\PDOException $e) {
    die("Unable to query the database: ".$e->getMessage()."\n");
}

/*************************** output what we've got ****************************/
echo("<"."?xml version=\"1.0\" encoding=\"utf-8\"?".">\r\n");
?>
<rss version="2.0">
<channel>
 <title>FreeGameDev</title>
 <link><![CDATA[<?php echo($config["url"]); ?>]]</link>
 <description>Free Game Development</description>
<?php foreach($rows as $row) { ?>
  <item>
    <title><![CDATA[<?php echo(htmlspecialchars($row["title"])); ?>]]></title>
    <description><![CDATA[<?php
        echo("by ".htmlspecialchars($row["username"])." ");
        $t = $row["t"] + 0;
        if($t < 60)
            echo("just now");
        else if($t < 60*60)
            echo(($t/60)." minute".($t/60 > 1 ? "s" : "")." ago");
        else if($t < 60*60*24)
            echo(($t/60/60)." hour".($t/60/60 > 1 ? "s" : "")." ago");
        else
            echo(($t/60/60/24)." day".($t/60/60/24 > 1 ? "s" : "")." ago");
        ?>]]></description>
    <guid><![CDATA[<?php echo(
        /* create a link to the discussion. Slug might contain the id already */
        $config["url"]."/d/".$row["id"]."-".$row["id"]."-".
        htmlspecialchars($row["slug"]));
        ?>]]</guid>
    <pubDate><![CDATA[<?php
        echo(htmlspecialchars($row["d"]." ".date("T"))); ?>]]></pubDate>
  </item>
<?php } ?>
</channel>
</rss>

Let me know if it works, or if you have any questions, etc. Code is public domain or CC0 whichever you prefer, you're free to use it on freegamedev.net.

Cheers,
bzt
User avatar
bzt
 
Posts: 332
Joined: 23 May 2021, 21:46

Re: [Meta] Proposed FreeGameDev changes

Postby Julius » 06 Jun 2021, 23:17

bzt {l Wrote}:Oh, and by the way, taking a look at the sql schema revealed that Flarum is storing the user's IP address with every post without the user's consent which is in a direct violation of GDPR. You should do something about it, turn this feature off if possible.

Yes I am aware of this and also not sure why this is hard-coded in Flarum. I asked one of their developers and their suggestion was to periodically overwrite it with a script or so :eew: I'll probably file a bug report and see where this goes. However, it is not in direct violation of the GDPR, but indeed I need to include this in the privacy policy that users have to consent to (which I haven't got around adding to the forum yet).

bzt {l Wrote}:Okay, here's my ca. 50 SLoC version...


Thanks a lot! This looks like a simple initial solution indeed, but I need to study the code a bit closer. Two quick questions though: 1. I assume you intend to just run this periodically with a cron? 2. How can I best change the output location of the resulting .xml? Because in Flarum only the "public" sub-directory is accessible via the webserver and this script looks like something that I would prefer to have outside of that directory.

Edit:

bzt {l Wrote}:One thing I couldn't figure out, is if the discussions table's slug field contains the sluggified title only, or the entire url part with the discussion id as well

It contains the sluggified title only.
User avatar
Julius
Community Moderator
 
Posts: 3297
Joined: 06 Dec 2009, 14:02

Re: [Meta] Proposed FreeGameDev changes

Postby bzt » 07 Jun 2021, 10:34

Julius {l Wrote}:Yes I am aware of this and also not sure why this is hard-coded in Flarum. I asked one of their developers and their suggestion was to periodically overwrite it with a script or so :eew: I'll probably file a bug report and see where this goes. However, it is not in direct violation of the GDPR, but indeed I need to include this in the privacy policy that users have to consent to (which I haven't got around adding to the forum yet).
Ok, I see you're aware. What violates GDPR is that it is storing without user's consent or knowing. Letting them know isn't a good a solution, just a step in the right direction, but makes it GDPR-compatible.
A proper solution would be a one line patch in ProcessIp.php at line 22:
{l Code}: {l Select All Code}
-  $ipAddress = Arr::get($request->getServerParams(), 'REMOTE_ADDR', '127.0.0.1');
+  $ipAddress = '127.0.0.1';
This will solve the issue without the need to touch other parts of the codebase.

Julius {l Wrote}:Thanks a lot! This looks like a simple initial solution indeed, but I need to study the code a bit closer. Two quick questions though:
1. I assume you intend to just run this periodically with a cron?
No, this is supposed to be called as a cgi by the webserver.

Julius {l Wrote}:2. How can I best change the output location of the resulting .xml?
The xml is outputted to the stream sent to the client, no local files created. This is a read-only code, it does not modify the database neither any server files (it just reads the config, that's all, it does not touch anything else).
Julius {l Wrote}:Because in Flarum only the "public" sub-directory is accessible via the webserver and this script looks like something that I would prefer to have outside of that directory.
Using "public" is quite common, and you definitely want this inside, because that's how it supposed to work (it's minimal, has no url routing, so it must be the webserver that serves it when "rss.php" url is called). Storing the xml in a file won't work as it has to serve up-to-date data and it contains time specifications like "just now" or "2 minutes ago" relative to the time of the http query, which would be impossible with a cached file.

If you have any doubts what it's doing, just ask, I'll answer, but it is extremely simple and secure:

1. It includes the Flarum configuration to get table prefixes and connection credentials. There's no controller part, it does not parse urls neither does any routing. As simple as it gets. (You could add a "/rss/" to "rss.php" rewrite rule to Apache if you want)

2. Lines from **** connect to **** get some data comments connects to the database.

3. Lines between **** get some data and **** output comments queries the database but it does not use any user provided strings for that, therefore no sql injection possible (uses only strings from the Flarum config, but nothing from the end user or from the Apache environment). You could say this is the "model" part.

4. Finally, lines after **** output comment is the view, which just outputs rss xml with a loop. This part contains no logic, just string formatting, and the database connection is already closed when code execution gets here.

Julius {l Wrote}:
bzt {l Wrote}:One thing I couldn't figure out, is if the discussions table's slug field contains the sluggified title only, or the entire url part with the discussion id as well

It contains the sluggified title only.
Then printing the id is necessary. Thanks! Oh, and I've use htmlspecialchars there too, my mistake, please replace it with urlencode($row["slug"]) as the guid field contains an url not html tags.

Cheers,
bzt
User avatar
bzt
 
Posts: 332
Joined: 23 May 2021, 21:46

Who is online

Users browsing this forum: No registered users and 1 guest

cron