Here is my proposal for a working implementation of albods. The current requeriments is: 1. It must be backwards compatible without the need for LD_PRELOAD hacks, but should be able to use them 2. For some file formats (like mp3), the data part is the only important one; for some others (e.g. a new albod-only fileformat) the albod should be traeted as a single unit for non-albod-aware programs 3. The albod must be represented as a flat directory; however, we have to think hard about albods-within-albods and compatibility issues. Also, it must have a 'packed' representation and should be able to convert between them on the fly. The packed representation should be readable by standard unix tools. Also, to confuse a bit more, it can have subdirs which can pack/unpack on the fly like itself (so it can still be treated like a flat directory). [FIXME: any other ideas?] So I planned this structure: - Albod is filename.albod, e.g., my_song.mp3.albod - Data part, if using compatibility mode, is my_song.mp3 - Dot files within the albod are reserved - Applications should agree to what should be the name of their subparts The structure is this: my_song.mp3.albod/ .options data icon.xpm my_song.mp3 -> hardlink to my_song.mp3.albod/data Hard links generate maximum compatibility, but causes some randomness if they are deleted. My current idea is to ignore it to avoid confusion and creating an utility to fix the albod manually. Here are the explanations for the files: my_song.mp3.albod/ - Treated as a file called my_song.mp3 by albod aware apps .options - Contains (without quotes, one per line): MUST BE THE FIRST ELEMENT WHEN PACKED! FIRST LINE IS ALWAYS "ALBOD=6518F161" (file magic)! "compat=data" if the compatibility link should be a hard link to the 'data' part It is used for already existing formats like .mp3 "compat=dir" if the compatibility link should be a symlink to the whole albod dir (we can't hardlink directories) Used when you want to treat the albod as a dir when using old apps "compat=albod" if the compatibility link should be a file which is the whole albod in packed form (treated as compat=none if there is no LD_PRELOAD hack running) Used when you want to be able to 'cp' a whole albod. This is the best idea for new apps. "compat=none" means there is no compatibility link and that non-albod apps should not be doing anything with this albod Used when you're picky and thinks legacy apps should not be used in your shiny new file. Also used when the LD_PRELOAD hack is not available. DEFAULT: compat=none "pack=always" means it should be unpacked on the fly by the lib and should not be stored as a normal dir This uses less disk space and is slower. This also makes compat=albod work without LD_PRELOAD hacks. "pack=auto" means it can be stored as a dir and will be autounpacked to a dir (replacing the packed file) This uses less disk space but requires a lot of it while unpacking. This is used so that you can copy an albod as a file by some random app and it will be unpacked to its normal form when some albod-aware lib touches it. The autounpack option controls the way it is done. "pack=never" means it _should_ be stored as a dir. A program needs override flags to force packing anyways. It can read it as a file if it wants, but when created from file will fully unpack. With pack=auto, a cron job/disk packing daemon/ user requested pack-all-my-dirs-because-I-need- more-space can repack the albod. This option is not intended to be used except in some very special cases. DEFAULT: pack=auto "autounpack=full" means that when albod is opened it is fully unpacked if it was packed. The original file is replaced by the dir. This avoids the loss of disk space caused by constant unpacking and is faster. "autounpack=auto" means that only the acessed files will be unpacked. After all are acessed (and unpacked), the file will be replaced by the dir. Faster startup than autounpack=full, but uses a lot of disk space before it gets fully unpacked. "autounpack=partial" Like autounpack=auto, but files will be repacked after being closed. The dir will never apear unless explicitly requested. Faster than pack=always, and uses less space than autounpack=auto. Opens the whole albod faster than autounpack=full. "autounpack=never" Like autounpack=partial, but do not allow the dir to appear even if requested. Used when you want the faster speed of pack=auto but do not want to allow it to appear as a dir. DEFAULT: autounpack=full "version=0" If version is _not_ zero, the program must report incompatible file formats. Used for forward compatibility. DEFAULT: version=0 "mimetype=application/octet-stream" MIME type for this albod's 'data' part. If it is "application/x-albod" it means this albod's data is understood only by albod-aware apps. DEFAULT: if compat=data mimetype=application/octet-stream else mimetype=application/x-albod Note that compat=data won't make sense with pack=always or autounpack=never. Also autounpack makes no sense with pack != auto. .lock - Subdir with one item per item in the root of the albod. If it is present here, it has the FQDN:PID of the program using this item (part/dotfile). This subdir won't appear in packed albods, and is the only item that can't be locked (for obvious reasons). data - The main data part. Required if compat=data or mimetype is not albod. Optional if compat != data and mimetype==application/x-albod or x-albod-subdir. data.* - Application-specific parts start with data. . icon.xpm - An example of a non-application-specific part. A icon. Completely optional. .my_song.mp3.albod - A packed albod. It is a normal file, not a dir. .my_song.mp3.albod.tmp/ - The partially-unpacked albod dir .my_song.mp3.albod.lock - This locks the whole albod. Used when messing with the directory/packed file structure. It contains FQDN:PID. my_song.mp3 - This is the compatibility link to: my_song.mp3.albod{,.tmp}/data (hardlink); or my_song.mp3.aldob{,.tmp}/ (symlink); or my_song.mp3.albod (symlink to file); or nothing (it isn't there). If any of the content parts (data, data.* and the non-app-specific parts) is a subdir, it is treated like an albod (the .options file is inherited with the following changes: compat=none, and mimetype for compat=none is application/x-albod-sudir. These can be overriden). When an albod-aware wants to open an albod, it calls the albod lib with the filename without the extension. This means: 1. If my_song.mp3.albod exists, it does its part (magic unpacks/whatever) and returns the whole albod; 2. If it doesn't but my_song.mp3 exists, it creates a 'virtual albod' in memory with only the 'data' part (the default values for the options file are right in this case, since compat=data and mimetype=application/octet-stream). If the albod is changed (not the locks or the data), a real albod is created (holding the main albod lock, creating the dir and hardlinking the data part, creating the changed file, getting the lock on it and releasing the albod global lock) The packed albod is just an ar file. If the albod has other albods within it, they will be packed first. If needed, a dir called .my_song.mp3.albod.pack and its lockfile (.my_song.mp3.albod.pack.lock) will be created to hold temporary files. Albods _must_ have .options files. If it doesn't or its magic is invalid it is treated as a normal dir/file. Files within an albod are treated like files outside an albod for albod-aware apps, that is: - If my_song.mp3.albod exists, treat it as my_song.mp3 and ignore the real my_song.mp3 if it exists - If my_song.mp3 exists but my_song.mp3.albod doesn't, treat it like a default albod (like when opening it) - dotfiles are ignored (this is to be consistent with what happens inside an albod). - Normal dirs are treated like subdir albods (compat=none, mimetype x-albod-subdir). [FIXME: s/exists/exists and is valid/ above] [FIXME: explain this mess better]