Dotfile Madness

Dotfile Madness

Postby dulsi » 16 Jun 2019, 11:57

LWN had a quote from this blog. I did not know of XDG Base Directory Standard. While I don't consider the proliferation of dot files to be a problem, the standard is a good idea. I'll have to incorporate that into my games. I had started creating .identical to store information for my games but Troll Bridge still uses .troll.
dulsi
 
Posts: 345
Joined: 18 Feb 2016, 15:24

Re: Dotfile Madness

Postby fluffrabbit » 16 Jun 2019, 13:08

TBH I don't know where to find my settings and saves, but in my user folder I have quite a few dotfolders for various games. You can't expect all those necrotic developers to fix the situation.
fluffrabbit
 
Posts: 570
Joined: 11 Apr 2019, 11:17

Re: Dotfile Madness

Postby Lyberta » 17 Jun 2019, 08:38

Obligatory implementation in C++:
{l Code}: {l Select All Code}
/// \file
/// \brief Source file that contains implementation of the POSIX FileSystem
/// class.
/// \author Lyberta
/// \copyright GNU GPLv3 or any later version.

#include <ftz/Platform/POSIX/FileSystem.h>

#include <unistd.h>
#include <sys/types.h>
#include <pwd.h>

namespace ftz::Platform::POSIX
{

using namespace std::string_literals;

std::filesystem::path FileSystem::GetUserDirectory()
{
   auto env = std::getenv("HOME");
   if ((env != nullptr) && (*env != '\0'))
   {
      return env;
   }
   auto passwd = getpwuid(geteuid());
   if (passwd == nullptr)
   {
      throw std::runtime_error{"FileSystem::GetUserDirectory: "
         "Unable to get directory."};
   }
   return passwd->pw_dir;
}

std::filesystem::path FileSystem::GetUserDataDirectory()
{
   auto env = std::getenv("XDG_DATA_HOME");
   if ((env != nullptr) && (*env != '\0'))
   {
      return env;
   }
   auto path = GetUserDirectory();
   path /= ".local/share"s;
   return path;
}

std::filesystem::path FileSystem::GetUserConfigDirectory()
{
   auto env = std::getenv("XDG_CONFIG_HOME");
   if ((env != nullptr) && (*env != '\0'))
   {
      return env;
   }
   auto path = GetUserDirectory();
   path /= ".config"s;
   return path;
}

std::filesystem::path FileSystem::GetUserCacheDirectory()
{
   auto env = std::getenv("XDG_CACHE_HOME");
   if ((env != nullptr) && (*env != '\0'))
   {
      return env;
   }
   auto path = GetUserDirectory();
   path /= ".cache"s;
   return path;
}

std::filesystem::path FileSystem::GetUserRuntimeDirectory()
{
   auto env = std::getenv("XDG_RUNTIME_HOME");
   if ((env != nullptr) && (*env != '\0'))
   {
      return env;
   }
   throw std::runtime_error{"FileSystem::GetUserRuntimeDirectory: "
      "No directory is set."};
}

}


I consider any software that writes directly into home to be crapware. Examples: bash, less, X11, ALSA, Git, wget, PulseAudio.

I think the only way to sanely use Linux is to put each program in its own container so it only shits itself. Too bad we are not there yet.

EDIT: Wait, no. I'm so retarded. The only sane way is to use non-Unix/POSIX/Linux-like system and put Linux into VM or container. Linux was never sane to begin with.
User avatar
Lyberta
 
Posts: 648
Joined: 19 Jun 2013, 10:45

Re: Dotfile Madness

Postby fluffrabbit » 17 Jun 2019, 09:08

You're including a lot of POSIX-specific stuff. I'm not trying to play devil's advocate, but surely there's a way to uniformly access an application-specific folder on Windows, Android, iOS, etc. Even Web Storage can probably function as a virtual user settings folder if filesystem emulation (see: Emscripten) is used.

EDIT: Wait, no. I'm so retarded. The only sane way is to use non-Unix/POSIX/Linux-like system and put Linux into VM or container. Linux was never sane to begin with.

You edited right before I decided to post. Yeah, don't use Linux-specific stuff. A VM though, I dunno. There is a video on YouTube called The Thirty Million Line Problem in which the Handmade Hero guy rambles about ISAs and doing away with operating systems. Not 100% related, but when I think of VMs, Docker, etc. I cringe. That stuff is too bloated in current implementations.
fluffrabbit
 
Posts: 570
Joined: 11 Apr 2019, 11:17

Re: Dotfile Madness

Postby Lyberta » 17 Jun 2019, 09:27

fluffrabbit {l Wrote}:You're including a lot of POSIX-specific stuff. I'm not trying to play devil's advocate, but surely there's a way to uniformly access an application-specific folder on Windows, Android, iOS, etc. Even Web Storage can probably function as a virtual user settings folder if filesystem emulation (see: Emscripten) is used.


Because that's the POSIX version. You can see that class is in POSIX namespace. I have a cross platform dispatch code.

fluffrabbit {l Wrote}:You edited right before I decided to post. Yeah, don't use Linux-specific stuff.


It's hard to tbh. The thing that annoys me the most is FHS which is completely braindead. Thankfully, Debian recently adopted "usrmerge" so /bin /sbin and /lib* become symlinks into /usr/*. I'm looking forwards to removing those symlinks completely. Sure, it will break POSIX scripts that expect /bin/sh to exist, but who cares? User sanity is more important than some POSIX shit.

Next I hope they will rename /usr to something sensible. At least Windows has C:\Program Files. Tbh I really like Windows naming convention. And windows doesn't have dotfiles!
User avatar
Lyberta
 
Posts: 648
Joined: 19 Jun 2013, 10:45

Re: Dotfile Madness

Postby fluffrabbit » 17 Jun 2019, 10:10

Windows uses NTFS to hide things. Dotfiles are more robust because they are filesystem-agnostic. As for naming conventions, it's pretty subjective. I agree that /usr is a stupid name. It's not /home/username, it's a system folder. Why have a system folder that's pronounced "user" on a multi-user system? It's nuts.

I can see that you have tackled the platform abstraction problem head-on. I am not qualified to criticize any of that, but it looks cool. I don't search GitLab and I'm not fond of the GPL, so under normal circumstances I probably wouldn't take notice, but since you pointed it out I think that whatever FTZ does, it's a step in the right direction.
fluffrabbit
 
Posts: 570
Joined: 11 Apr 2019, 11:17

Re: Dotfile Madness

Postby sago007 » 18 Jun 2019, 20:15

I did write a C++ library to abstract the differences between the desktop operating systems: https://github.com/sago007/PlatformFolders
It goes a bit further and also implements the XDG user directories (Documents, Pictures etc) which seems to be something a lot of programs does not do properly. Java and Mono both returns $HOME when asking for the Documents folder.

I do think it is easier when implementing a new program. I can't imagine the amount of scripts that will be broken if ".ssh" moves and ".profile" is needed to actually define the environment. However I do not see why modern programs like Atom or TrenchBroom does not respect it.

For OpenArena I have been looking into moving away from ".openarena" but I have to keep a symlink for a very long migration period to not make all documentation out of date.
sago007
 
Posts: 9
Joined: 16 Jul 2017, 10:06

Re: Dotfile Madness

Postby fluffrabbit » 18 Jun 2019, 20:31

That's awesome! One point of confusion (not with these abstractions, but with the Linux way of doing things in general) is the difference between XDG_CONFIG_HOME and XDG_DATA_HOME. I assume they are both settings folders for the apps.
fluffrabbit
 
Posts: 570
Joined: 11 Apr 2019, 11:17

Re: Dotfile Madness

Postby GunChleoc » 18 Jun 2019, 20:37

Settings would go into config, other user data like savegames would go into data.
User avatar
GunChleoc
 
Posts: 440
Joined: 20 Sep 2012, 22:45

Re: Dotfile Madness

Postby sago007 » 18 Jun 2019, 21:09

If in doubt I would put it into XDG_DATA_HOME. I would only put things into XDG_CONFIG_HOME if the files are meant to be human editable. On Windows I map both into Roaming.
I see XDG_CONFIG_HOME as more useful for tools than games. Most games will need a data folder and it might make sense to keep all user files together.

On Windows there seems to be some confusion between "Roaming" and "Local". I map XDG_CACHE_HOME to "Local". I see "Local" as the cache/log dir for data that does not need back-up.
sago007
 
Posts: 9
Joined: 16 Jul 2017, 10:06

Re: Dotfile Madness

Postby fluffrabbit » 19 Jun 2019, 04:31

Confusion is definitely the problem. I would want to keep savegames with settings for a game. I have no idea about "Roaming" and "Local". Ideally the folder setup on every computer in the world would just be .cache and .appdata, and that's it.
fluffrabbit
 
Posts: 570
Joined: 11 Apr 2019, 11:17

Re: Dotfile Madness

Postby dulsi » 09 Jul 2019, 17:02

Updated Troll Bridge to use XDG_DATA_HOME. It automatically copies the old save files to the new location. It doesn't delete the old .troll directory in case of problems.

A nice side benefit of doing this is that I noticed boost::filesystem functions are in the C++17 standard. The randomization code was added in C++11 so Troll Bridge no longer needs boost. I then put boost back in for the Windows version because the cross compiler on Fedora is older and doesn't support the filesystem header. I did lose the boost randomization code and the filesystem functions are similar enough I was able to just use a namespace alias at the start of the file instead of two implementations.
dulsi
 
Posts: 345
Joined: 18 Feb 2016, 15:24

Re: Dotfile Madness

Postby fluffrabbit » 09 Jul 2019, 17:54

Great, but AFAIK the C++17 filesystem API is mostly just good for enumerating files, dealing with symlinks, etc. I haven't used it yet. In the case of Linux, XDG_DATA_HOME should be an environment variable accessible from plain old C, and pre-C++17 streams could be used for files. Windows uses environment variables as well. What I'm hinting at is that more cross-platform support without the need for newer compiler features or large libraries is a good design goal.
fluffrabbit
 
Posts: 570
Joined: 11 Apr 2019, 11:17

Re: Dotfile Madness

Postby dulsi » 09 Jul 2019, 18:26

fluffrabbit {l Wrote}:Great, but AFAIK the C++17 filesystem API is mostly just good for enumerating files, dealing with symlinks, etc. I haven't used it yet. In the case of Linux, XDG_DATA_HOME should be an environment variable accessible from plain old C, and pre-C++17 streams could be used for files. Windows uses environment variables as well. What I'm hinting at is that more cross-platform support without the need for newer compiler features or large libraries is a good design goal.

I'm not trying to make a solution for everyone. If that is what you want, seems like sago007's PlatformFolders may be a good solution.
dulsi
 
Posts: 345
Joined: 18 Feb 2016, 15:24

Re: Dotfile Madness

Postby fluffrabbit » 09 Jul 2019, 18:34

dulsi {l Wrote}:
fluffrabbit {l Wrote}:Great, but AFAIK the C++17 filesystem API is mostly just good for enumerating files, dealing with symlinks, etc. I haven't used it yet. In the case of Linux, XDG_DATA_HOME should be an environment variable accessible from plain old C, and pre-C++17 streams could be used for files. Windows uses environment variables as well. What I'm hinting at is that more cross-platform support without the need for newer compiler features or large libraries is a good design goal.

I'm not trying to make a solution for everyone. If that is what you want, seems like sago007's PlatformFolders may be a good solution.

It is a good solution, but for a game I probably wouldn't even use that, I'd try to cram the bare minimum functionality into 200 lines of code or less. You used to support DOS. I'm not saying you should, but I'm on kind of an oldschool kick right now, I dunno.

Related: High-level infrastructure may be doomed.
fluffrabbit
 
Posts: 570
Joined: 11 Apr 2019, 11:17

Re: Dotfile Madness

Postby sago007 » 09 Jul 2019, 22:28

Another thing that does annoy me is that using the environment variables on Windows fails for international characters. This may be better in the future now that Windows 10 finally supports UTF-8 but I still think it will take a while for all the legacy systems to reach EOL.

At the moment you need the wide characters API on Windows which prevents most of the basic C and C++ libraries.

I use SDL2 and PhysicsFS to handle files and system calls across platforms. They both converts all char-strings to UTF-8.
sago007
 
Posts: 9
Joined: 16 Jul 2017, 10:06

Re: Dotfile Madness

Postby fluffrabbit » 10 Jul 2019, 00:19

Can't you use C++ utf16 strings or a vector of uint16_t? All you have to do is convert your utf8 strings to codepoints and you're good to go. Since enumerating files from the filesystem is less common, you probably only need one-way conversion. If things are more fiddly than that, maybe double up the characters by doing reinterpret_cast<char*>( MyWideString.data() ) or something similar.
fluffrabbit
 
Posts: 570
Joined: 11 Apr 2019, 11:17

Re: Dotfile Madness

Postby Lyberta » 10 Jul 2019, 08:53

fluffrabbit {l Wrote}:Can't you use C++ utf16 strings or a vector of uint16_t? All you have to do is convert your utf8 strings to codepoints and you're good to go. Since enumerating files from the filesystem is less common, you probably only need one-way conversion. If things are more fiddly than that, maybe double up the characters by doing reinterpret_cast<char*>( MyWideString.data() ) or something similar.


What the hell? That doesn't make any sense. If you don't care about full Windows support, you can just store string internally in UTF-8 form. C++20 has char8_t and std::u8string but that is not great at all. As of C++20 you'd better write your own Unicode library. In fact, in any version of C and C++ so far you'd better write your own Unicode library since there is pretty much zero support in all standards.

So for Windows you convert from UTF-8 to UTF-16, then copy result into NUL-terminated sequence of wchar_t and then feed it to the WinAPI you need. That is not full support because NTFS and WinAPI allow unpaired surrogates in filenames which are ill-formed Unicode. But I'd say this doesn't matter for games because you can't even print ill-formed string.

In any case, C++17 std::filesystem is enough for basic FS operations, but there is no way to read and write Unicode to files without 3rd party library.
User avatar
Lyberta
 
Posts: 648
Joined: 19 Jun 2013, 10:45

Re: Dotfile Madness

Postby Julius » 11 Jul 2019, 17:25

Moderator notice, split off UTF discussion.
User avatar
Julius
Community Moderator
 
Posts: 2684
Joined: 06 Dec 2009, 14:02

Who is online

Users browsing this forum: No registered users and 0 guests