LITTLEBLACKDOG.COM

http://www.osnews.com/story.php?news_id=6101

according to this article, it looks like Apple is adding metadata support ala BeFS file attributes to Finder. in BeOS you could use these attributes to store, say song artist, title, track length, etc. of an mp3 right in the file system, then use normal file system searches to sort and find the songs you wanted. it was also used to store and sort email, the actual email message would be the file, but the sender information as well as replied to, forwarded, etc. would be stored in the file system. lots of other nifty things along those lines were implemented, including, recently IM buddy lists.

so what does this mean? hopefully fast indexing of directories. a desktop and folders that displays icons when they are created, not when they are forced too. and awesomely detailed and fast file searches.

and of course bragging rights. if Apple pulls this off for 10.4, scheduled for release at the end of the year, they will be at least two years ahead of M$ on yet another highly touted feature of longhorn (with what seems to be a less bloated approach as well).

this is all speculation of course, but it will be interesting to see how it plays out.

Posted: Fri Feb 20, 2004 10:08 pm Post subject:

With regards to file systems, you ever notice in most *nix filesystems how long it takes to draw each icon when you go into a directory that has hundreds to thousands of files? Longer than it is in Windows IMHO.

I'm a bit torn by all this talk of "journaled" filesystems and "meta data". One one side there's people whining about FS bloat and how hard drive space gets eaten up too quickly. Then there's the other camp that complains about lack of features in the FS, such as journaling, meta data, indexing. In some cases (Linux, *BSD) we get to choose which FS and thus what features, we want to use. But with Windows and MacOSX we're pretty much stuck with what comes in the box.

So what's my point? I don't know, I have a cold and I'm delirious. On top of that I've been coding for some time now. And on top of that again, I haven't had the same dosage of caffine + niccotine I'm use to.

Posted: Sat Feb 21, 2004 6:11 am Post subject:

i'm going to make an argument for journalling and atomic file systems. reiser is atomic and journalling and its four times faster then ntfs.

i think the problem with most *nix's isn't their file systems not being fast, but their directory browsers not being fast. doing an ls in a command line will start spitting out the list of files in no time at all, but try to do that in Finder or Konqueror, and it takes a while.

my guess is whatever algorithms they use to index the directory, determine icons, icon placement, label color, displaed name, extra icon attributes (like symlink arrows, italic text, whatever), and then the actual drawing of those is what slows the process down. its definately not an O(n) function in Finder, i'm not sure about Konqueror, i don't play with it enough anymore.

reading some of the comments from that article, people seem to think that HFS+ already has support for extended attributes, its finder that can't do anything with them. people also claim that there are a lot of ex-Be employees working at apple now, so i can only imagine each new point release will move closer and closer to computing utopia. (at least that's what i'd like to imaging).

Moderator Joined: 26 May 2001 Posts: 8155 Location: Borneo

Windows just checks the extention but under *nix you have to head the first few lines of a file and run file on it to see what kind of file it really is. That just takes longer.

Moderator Joined: 26 May 2001 Posts: 8155 Location: Borneo

The guy who developed BeFS works now for apple so it is not surprising that they are comming out with a good FS. Also the apple filesystems always supported some kind of meta-data so it isn't that surprising that they come out with a filesystem that supports that.

Moderator Joined: 26 May 2001 Posts: 8155 Location: Borneo

Lycander wrote:
. In some cases (Linux, *BSD) we get to choose which FS and thus what features, we want to use. But with Windows and MacOSX we're pretty much stuck with what comes in the box.

On windows you can choose more FS than only NTFS and VFAT except it cost a lot of money to do so

Posted: Sun Feb 22, 2004 5:41 am Post subject:

csign wrote:
The guy who developed BeFS works now for apple so it is not surprising that they are comming out with a good FS. Also the apple filesystems always supported some kind of meta-data so it isn't that surprising that they come out with a filesystem that supports that.

specifically the old HFS and HFS+ stored file information in a hierarchy that extended to the file structures themselves. Files have what are known as resource forks, and the resources in those forks were objects, many of which were common and helped the MacOS be as consistent as it has always been. I still have my copy of Zen and the Art of Resource Editing for reference although it's not really relevant for OSX anymore.

A journaling file system like BeFS is another kettle of fish altogether. Digging out my O'Reilly copy of Programming the Be Operating System, I can see that BeFS is also heirarchical and represents files as nodes in the heirarchy. But built into the file system itself is the journal, and like HFS there is a concept of extended attributes. But unlike HFS, there could be multiple attributes, and they were extensible at the file level. BeFS also had built in indexing and a query interface. All attributes were indexable and were indeed indexed and the whole indexing mechanism had (has?) its own API. The journal itself is elegant and simplistic. Each file operation is journaled and the transaction is written to a log. Upon reboot, the log is played back, therefore any incomplete transactions are then either rolled back or completed. The other thing the BeFS is good for is streaming. As a multimedia OS, BeOS was 10 years ahead of itself, as the Amiga was in its day.

Journaling isn't that hard to implement, if you want to keep things simple. Since there is nothing simple about NTFS though, implementing journaling into it has been a lot of work. HFS+ shouldn't be that hard to deal with, especially since the Mach kernel is modular enough. Times like this, when you compare different operating systems you can see what an abomination Windows is, but you can also see the pros and cons of all operating systems.

Moderator Joined: 26 May 2001 Posts: 8155 Location: Borneo

The filesystem stays intact because of the log but i don't think it can repair the file.

Posted: Sun Feb 22, 2004 1:28 pm Post subject:

csign wrote:
The filesystem stays intact because of the log but i don't think it can repair the file.

no, the file either is ok because the log says so, reverts to the previous revision because the log doesn't say so. The main thing is you don't have one of those "this operation system was shut down improperly, it now has to check the file system consistency for n minutes" which I might note is both a Windows and *nix trait. Apple file systems you have to do the disk check by hand using Disk Doctor. Except now that we got Mach, it does the fsck at boot. Just like any other unix.

so as you said either way the filesystem is intact. That's what we like about journaling.

Moderator Joined: 26 May 2001 Posts: 8155 Location: Borneo

I think your misstaken.The file may not be correct but the filesystem is. For the file to be correct you need another kind of loging filesystem.

Posted: Sun Feb 22, 2004 6:59 pm Post subject:

HFS+ already has journalling (as of panther).

according to practical file system design:

some smart guy (aka Dominic Giampaolo) wrote:

To understand journaling, we first need to understand the problem that it tries to solve. If a system crashes while updating a data structure on disk, the data structure may become corrupted. Operations that need to update multiple disk blocks are at risk if a crash happens between updates. A crash that happens between two modifications to a data structure will leave the operation only partially complete. A partially updated structure is essentially a corrupt structure, and thus a file systemmust take special care to avoid that situation.
.
.
.
Journaling, a technique invented by the database community, guarantees the correctness of on-disk data structures by ensuring that each update to the structure happens completely or not at all, even if the update spans multiple disk blocks. If a file system uses journaling, it can assume that, barring bugs or disk failure, its on-disk data structures will remain consistent regardless of crashes, power failures, or other disastrous conditions. Further, recovery of a journaled file system is independent of its size. Crash recovery of a journaled volume takes on the order of seconds, not tens of minutes as it does with large nonjournaled file systems. Guaranteed consistency and speedy recovery are the two main features journaling offers.

journalling is just like database transactions. either the data gets written, or it doesn't. as long as file writes happen within a single transaction, you can be gauranteed your file is correct. if it crashes int the middle of the transaction, your file may not exist or may still be the previous version of the file, but you should never get a corrupted file from a system crash. and no incomplete transactions also means no corrupted file system.

Moderator Joined: 26 May 2001 Posts: 8155 Location: Borneo

data structure on disk = filesystem
A file can be corrupted in a journaled filesystem but the filesystem itself will be correct.

Posted: Mon Feb 23, 2004 5:00 am Post subject:

csign wrote:
data structure on disk = filesystem
A file can be corrupted in a journaled filesystem but the filesystem itself will be correct.

oops, you are totally right. i guess i should have read on. man, that's a great book.

it's available for free download from rentzsch: http://rentzsch.com/links/practicalFileSystemDesignPDF

*edit* page 115 (page 125 of the pdf), section 7.4 has the information about what is stored in a journal */edit*

Posted: Mon Feb 23, 2004 9:49 am Post subject:

csign wrote:
data structure on disk = filesystem
A file can be corrupted in a journaled filesystem but the filesystem itself will be correct.

One of the nicer aspects of the BeFS was that file operations were journaled. This is not to say that without effort you can corrupt a file on a disk - some people can do anything. But since file operations were journaled, file corruption within the file system is minimised.

a good link I just found is http://www.faqs.org/docs/Linux-HOWTO/Filesystems-HOWTO.html seems to cover every filesystem on the planet.