Writing neighboring files

[Mikenux]

This is the equivalent of #463, but for writing files.

To be exact, it is about writing/creating folders or files after the opening or creation of a folder or file.

There are already 4 app cases listed:

• Firefox, saving a webpage. An html file is saved (e.g. site.html) and Firefox automatically saves several files in a folder with the same name as the html file with a suffix (e.g. site_files). The html file and its companion folder are saved in the same folder.
• Inform. It creates a .materials folder (folder extension) with the same name and in the same folder as an open .inform folder. It also creates folders and files in the .inform project folder after creating it.
• Metadata Cleaner. Its developer wants to save copies of open files (to keep the originals) in their respective parent folders. Filenames should be the same as the originals but with a suffix (for example, filename.cleaned.ext, where ext is the extension of the original file).
• KiCad7. It asks for a project name when creating a project. This creates a folder with the name of the project, and in it files with the same name but with the extensions .kicad_pro, .kicad_pcb and .kicad_sch.

Cases, said another way:

• Save a file, then write a folder with multiple files in the same folder. The saved file and the written folder (partially) share the same name.
• Save/create a folder, then write other folders and/or files in it. They share (in part) or not the same name.
• Open a folder with extension, then create another folder with the same name but different extension in the same directory.
• Open files, then write files sharing the same name with a suffix and the same extension. Files can come from different locations.

Other cases are welcome. If they are different from those mentioned, please provide a description if possible.

Addition (June 18, 2023)

Other cases :

• IDEs must write files that are built in a project folder or a specific folder.
• Inform wants to create the .materials folder if it does not exist after opening the .inform folder.

[Mikenux]

For creating and saving multiple files:

• Case Examples: Inform and KiCad which create default files and folders when creating a project, Firefox which saves several files and folders.
• The number of files to be created or saved and their sizes, including those of folders, are displayed in the file chooser. A list with the details must be accessible.
• The project or website name is named without any extension.
• Note: The information must be accurate regarding the files and folders that will be created or saved (including saved from the Internet).
• Sample UI: SaveMultipleFiles

Feasible? Opinions?

[ctlcltd]

Interesting, but I don't think it's feasible. Usually the file chooser returns a location. As things stand, I don't think it's possible to pass much information. Such a mechanism involve the system api rather than the UI, would this mean modifying every existing file chooser? A few days ago I was seeing Gtk file chooser, I don't know if there are hooks to do this or make it possible with portals api.

[Mikenux]

I don't know because I'm not a developer. But this will certainly involve modifying the way file choosers work.

[Marukesu]

There's already the SaveFiles method in the FileChooser portal, the application send the list of files it want written and the portal will return the ones that got actually written.

[Mikenux]

@Marukesu: Indeed, but does it work by also creating folders? Also, the phrasing in the documentation leads me to think that the files are saved as is, with no option to change the name. Also, is file size taken into account?

Another question: What about the Firefox case? Are the relevant files of the website to save already downloaded or downloaded after saving the html file?

[Marukesu]

@Marukesu: Indeed, but does it work by also creating folders?

no, all the paths used will be exported to the sandbox as files.

Also, the phrasing in the documentation leads me to think that the files are saved as is, with no option to change the name.

yes, it's intended. if we are dealing with auxiliary files, like all the use cases said does, allowing to change the name won't be helpful.

Also, is file size taken into account?

no. why it would, the chooser is to select a location.

Another question: What about the Firefox case? Are the relevant files of the website to save already downloaded or downloaded after saving the html file?

i don't get it, the firefox case seem simple:

• firefox ask the user a name to save the site with.
• a SaveFiles call with [ 'site.html', 'site_files/file.txt', ... ] will return paths that can be used to write the contents.
• firefox write the files.

keep in mind that the portal doesn't save (or in the case of OpenFile, open) anything. it just return paths that can be used by the application, how and in what order things are written is up to the application.

[Marukesu]

expanding in the user cases:

• Inform and Metadata Cleaner are Portal to open file and neighbouring files #463 use cases, they want access to a path that came from a opened one.
• kicad case is similar to the firefox, a SaveFiles call with [ 'project-name/project-name.kicad_pro', 'project-name/project-name.kicad_pcb', 'project-name/project-name.kicad_sch' ] in the files option.

[Mikenux]

Although Inform and Metadata Cleaner are cases related to opening files/folders, I thought it was a good idea to address them separately so that #463 just focuses on the opening part, because the rest is about writing files. Note that I have not discussed these cases yet.

The first part offered is just when creating a project that also writes other files (KiCad) or an app that saves multiple files (Firefox). The goal is for the user to be aware of an app's writing activity (here: that other files are being written, which ones and how much storage space they take). This obviously requires extending the portal and file choosers.

[ctlcltd]

In my case the multiple files are saved on folder, but only the first file gets the right name, the other files get hashed file name, eg.

1. bouquet-right-filename.ext
2. -x--pf658akj.ext
3. --e-sfqxad-7.ext

Files are saved but are unrecognizable, except the first one that is saved. This was a month ago in Arch Linux, with xdg-desktop-portal-gtk, file chooser Gtk3, Qt6. Opening files from folder works as expected.

[Marukesu]

Although Inform and Metadata Cleaner are cases related to opening files/folders, I thought it was a good idea to address them separately so that #463 just focuses on the opening part, because the rest is about writing files. Note that I have not discussed these cases yet.

Unless the user set the chooser to "open in read-only mode" (availability depends of the backend), if you open a path you can already write to it. so any case that deals with "give access to a neighbour path based in a opened path" is a #463 case solely.

This issue only make sense if you are talking about "also create theses others files when creating this file".

The first part offered is just when creating a project that also writes other files (KiCad) or an app that saves multiple files (Firefox). The goal is for the user to be aware of an app's writing activity (here: that other files are being written, which ones and how much storage space they take). This obviously requires extending the portal and file choosers.

You can show the file names sent in the SaveFiles call already, that just need implementation changes in the backends to use that information.

The file size doesn't make sense at all with how the portal (or even any chooser API) works. Even if a option to send the file sizes is added, i don't think it will be used by libraries.

In my case the multiple files are saved on folder, but only the first file gets the right name, the other files get hashed file name, eg.

1. bouquet-right-filename.ext
2. -x--pf658akj.ext
3. --e-sfqxad-7.ext

Files are saved but are unrecognizable, except the first one that is saved. This was a month ago in Arch Linux, with xdg-desktop-portal-gtk, file chooser Gtk3, Qt6. Opening files from folder works as expected.

AFAIK, none of the more common libraries make use of the SaveFiles method, so a application will need to talk to the portal directly, or update the used library to make use of it.

[Mikenux]

@Marukesu:

Thanks!

For me, neighbor files are additional files to open (I made proposals for this which are now in the Discussions tab), and not provide access to the parent directory. But even if it would be to give access to the parent directory, in the case of Metadata Cleaner it is better to have something specific, rather than having access to the parent directory of each file (because the files can come from different locations).

[xhorak]

i don't get it, the firefox case seem simple:

* firefox ask the user a name to save the site with.

* a SaveFiles call with `[ 'site.html', 'site_files/file.txt', ... ]` will return paths that can be used to write the contents.

* firefox write the files.

keep in mind that the portal doesn't save (or in the case of OpenFile, open) anything. it just return paths that can be used by the application, how and in what order things are written is up to the application.

It's not that simple. Firefox asks for the filename by using filechooser, which SaveFiles does not allow to change. According to chosen filename it then creates the _files folder to store data.

Also there's a problem that SaveFiles does not deal with situation when the requested files already exists. The documentation says that the portal may ask to select different names (but there's no strong imperative that it will) and there are potentially dozens of files in that list in the _files subdirectory. Asking each one of them to select different name could be very annoying. Asking user to allow write access to the _files directory after the file chooser dialog sounds much better to me. There are more than pages to be saved - for example saving image, pdf, should not use the SaveFiles at all.

User can also choose to save the html only, which does not create the _files subdir. All is clear after the filechooser save dialog is closed.

[1] https://docs.flatpak.org/en/latest/portal-api-reference.html#gdbus-method-org-freedesktop-portal-FileChooser.SaveFiles

[xhorak]

For saving pages in case of firefox it should be sufficient to give access to the directory named after the selected filename from the file chooser. Having something like a #463 (comment) would be great.

[Mikenux]

For info, I move this to #1397.

@xhorak: The goal for me, in terms of UX, is to name the base file (like naming a project) and then save. Why do you want an extra dialogue? To not change the file saving is currently working (or possibly pass an additional option to the portal)?

[xhorak]

For info, I move this to #1397.

@xhorak: The goal for me, in terms of UX, is to name the base file (like naming a project) and then save.
Why do you want an extra dialogue? To not change the file saving is currently working (or possibly pass an additional option to the portal)?

That would be great. I could image that you would have some kind of special tailored case for the Firefox by passing portal that <filename>_files directory could be created (but it depends on the file type user picks - see the above screenshot). Problem is that it's not much general approach. Having second dialog when needed which states something like:

Firefox wants to write related the resources files into the /home/user/Downloads/some_page_files directory. Do you want to allow it?
Could work for Firefox case.

Or do you want to avoid additional changes on the application part beside than filechooser picker portal invocation or some other approach?

[Mikenux]

Don’t really know as I’m not a developper. I think that an app should adapt its request depending on what filter is selected or just sends the request (in Firefox case, telling which files to write) with some hint and file chooser adapt relative to selected filter.

[Mikenux]

Considering that this issue is old and that the discussion is almost 6 months old, I think I can look at the different cases to see what kind of user interface to expect and what can be grouped.

[Marukesu]

For the developer side, one way to do this would be:

1. add a 'neighbor' flag in the SaveFile method, that when used, gives the application the permission to write files relative to the returned one, excluding upper levels.

2. add a 'relative' flag in the SaveFiles method, that when used with some fd, if the application has already have the neighbor permission for that fd, just return the paths without showing a new dialog, otherwise show a dialog asking for permission to create neighbor files from the relative one. If the flag is not used, the behavior is the same as currently.

This way, Firefox would first usa a SaveFile call for the main file, with the possible filters and with the neighbor flag set, and if the selected filter is the one that save extra files, use a SaveFiles call to write them with the correct names.

[xhorak]

SaveFiles basically opens a directory chooser dialog, I guess that's not what we want. You can provide list of files in the SaveFiles call but still the dialog is shown. I could probably get the list with some extra effort on the Firefox side.

But changing the location in the SaveFiles dialog would pretty much breaks the saving code on the Firefox.

Or you mean all the work would be done by the portal depending what filter had been chosen? That's basically does not require any code change on the Firefox side, but it seems that's really customization just for Firefox on the portal side (I'm fine with that, but I guess not many are :) ).

[xhorak]

Anyway, @matthiasclasen you may have some ideas, at least according to some random quote on https://gitlab.com/fedora/sigs/flatpak/fedora-flatpaks/-/issues/13#medium-impact-issues :).

[ssokolow]

But changing the location in the SaveFiles dialog would pretty much breaks the saving code on the Firefox.

...not to mention all the applications where upstream gets the chooser automatically because they use Qt or GTK, may not even see Linux as a primary platform, and aren't really interested in Flatpak or Snap, so the package maintainers just have to go for something like --filesystem=home to keep people from faulting the Flatpak/Snap for being "broken" and moving on.

[Mikenux]

A second dialog is not excluded, as there is the case of the "Save Location" where the user will be asked to confirm the saving of the files (at least for now).

Otherwise, I guess some descriptive file that can be read by the portal might work (the thing I used for my other neighboring files proposals). However, this indeed depends on which cases other than Firefox can use it.

My main point is just to avoid two dialogs just to save one thing (the entire web page which is composed of multiple files). So depends on cases and development effort.

[ssokolow]

My main point is just to avoid two dialogs just to save one thing (the entire web page which is composed of multiple files). So depends on cases and development effort.

*nod* I can certainly agree on the "no two dialogs" part.

[Mikenux]

I don’t understand the thing on "changing the location in the SaveFiles dialog". What does this mean? Isn’t the user expected to choose the location, or that’s another thing?

[Marukesu]

SaveFiles basically opens a directory chooser dialog, I guess that's not what we want. You can provide list of files in the SaveFiles call but still the dialog is shown.

That's why the "relative" flag would be added, so that the portal have the necessary information to skip the directory chooser, or show a "create neighbor" dialog like @Mikenux suggested.

The, also added, "neighbor" flag in the SaveFile method , would alow backend implementers to inform that the application may write other files in the same directory or subdirectories.

[Mikenux]

@Marukesu: It is preferable to avoid two UXes for the same case.

By re-reading my proposal (the discussion I opened), write access is given when a file is saved by showing a file chooser (if the app has the focus), and is valid until the app is closed (because that’s difficult to externalize the save function). I wanted the user to have the opportunity to see at least which files will be saved, but I can forget that as the user is not necessarily aware of it in all cases. A system might also track that and probably show a write activity report to users, say when the app is closed. Maybe this is acceptable?

So:

• The app save one file.
• The app can save one list of additional files relative to the first saved file. Can we use some timing to return this list?
• If the app want to write another list of files (new files unrelated to the first), then a standard dialog is shown.

Here, no need for a flag or a specific dialog.

[ssokolow]

The app can save one list of additional files relative to the first saved file. Can we use some timing to return this list?

The current behaviour for mounting files into documents portal paths results in the provided file path being valid at least as long as the session and, if it's not marked as session-only, until explicitly revoked using the flatpak CLI.

(Try running flatpak documents --columns=path,origin,permissions on your system.)

I wouldn't want some "ToC file adjacent to the subresources" project-based tool like Eclipse or LyX or whatever to work initially and then unexpectedly start throwing errors about being able to write some files but not others when I go to Ctrl+S again later. That sounds like it'd be hell to diagnose and insanely frustrating for anyone who doesn't have it click that "Oh, this is how the documents portal works".

[Marukesu]

So:

• The app save one file.
• The app can save one list of additional files relative to the first saved file. Can we use some timing to return this list?

So, there's already the need of either a flag or a fully new method in the portal, because, IIRC, the application doesn't have access to the full host path of the file. So you need a way to say that the list of paths is relative to the specified file.

• If the app want to write another list of files (new files unrelated to the first), then a standard dialog is shown.

Here, no need for a flag or a specific dialog.

The alternative dialog covers the case of a application trying to save neighbor files when it doesn't have permission to do so, either because it didn't request, or the user revoked it afterwards.

So i wouldn't say it cover the same case of the choser (no relative file specified), nor the skip case (the user is aware and give the application permission to write neighbor files relative to the specified one).

[Marukesu]

I wouldn't want some "ToC file adjacent to the subresources" project-based tool like Eclipse or LyX or whatever to work initially and then unexpectedly start throwing errors about being able to write some files but not others when I go to Ctrl+S again later. That sounds like it'd be hell to diagnose and insanely frustrating for anyone who doesn't have it click that "Oh, this is how the documents portal works".

I would say the permission revoking for saving neighbors removes the ability of creating new files, not to writing to the ones already created from it.

[ssokolow]

Except that those sorts of project tools won't pop up a file picker the first time you choose something like File > Export > PDF (pdflatex). They'll just derive a filename from the existing file (eg. foo.pdf from foo.lyx) and try to create it.

...and so you'll get an error if you've been using the .dvi-based Ctrl+R preview and then, after said proposed timeout has run its course, ask to export a PDF or HTML bundle or what have you.

Same for an IDE where you've been doing debug builds that show up in the project folder for a while and then try to make a release build after the timeout has run its course, assuming it's an IDE which supports "multiple projects in a single folder" by expecting you to pick the project file instead of the containing folder.

It's just part of their design that they assume that permissions won't time out because that's not how POSIX normally works.

It's "Once you've 'Save As...''d once, you don't need to pick the file again when 'Save'-ing", except extended to newly created derived files from things like Export or Build too.

[Mikenux]

@Marukesu: I’m not assuming that the user will give permission to write additional files as we don’t necessarily know if these files are needed. So, I’m assuming to write these files without a permission. Even with what I said, given it is detectable (making the request within the timeout), I guess it is possible to make it a permission. Anyway, a flag can be used if needed. However, I don’t get why an app would forget to use the flag. Writing additional files is predictable, at least in the case of Firefox.

@ssokolow: Take a look at #1397. I’m working by case (like what I done for opening neighboring files), and I prefer to make things coherent. All cases don’t necessarily get the same treatment. For any new file or new group of files, except for exceptions, a dialog will be shown. Is that problematic if a dialog is shown asking the user where to save the file to export? Also, what’s the thing with the dvi? Is the app convert the lyx file to a dvi file in order to preview it? (Maybe that’s something for temporary files?)