> Anybody know where can I get the "atime" news server speedup patch for
> ext2?
> (not in www.ecsnet.com!)
I have a copy by coneill@premier.net (is this the same one?)
Also note that FreeBSD-current has them and a patch set for the latest
stable (2.1.5) is also available (email me for the article I saved).
Also some commercial NFS servers do (but not SunOS).
Someone else also made a patch set for Linux ext2 and INN & family
that uses inode numbers instead of path names for all the articles.
This speeds things up nicely for him, he says. I'm looking for my
reference to this, and I cannot find it! Where is it?! Who did this?
I really want to know if it speeds things up for him, as I personally
archive some newsgroups and the number of files in a single directory
gets rather large:
bash$ ls -ld .
drwxr-xr-x 2 news news 845824 Nov 1 05:00 .
bash$
So of course I'm interested in others' work on either making large
ext2 directories efficient (hashing, indexing, whatever) or using
efficient databases for USENET articles (someone mentioned modifying
INN to use their database (Oracle?) and reported enormous speedups,
and someone else mentioned using his own specialized flat file.)
Included are my sources as they relate to Linux ext2 atime update omission:
Path: news.panix.com!panix!bloom-beacon.mit.edu!gatech!news.mathworks.com!uunet!
in3.uu.net!news.fdt.net!fdtnet!yoda.fdt.net!fubar
From: Fu Bar <fubar@gnv.fdt.net>
Newsgroups: news.software.nntp
Subject: Re: atime updates on news spool disk - how about Linux?
Date: Wed, 16 Oct 1996 18:36:36 -0400
Organization: Florida Digital Turnpike
Lines: 12
Distribution: inet
Message-ID: <Pine.LNX.3.95.961016183554.29108j-100000@yoda.fdt.net>
References: <jdvicggo2t.fsf@terminus.cs.umb.edu> <53tus7$1p9@nexp.crl.com> <slrn
569r1a.dl.jason@pris.oit.co.uk> <54306t$lh9@news.xs4all.nl>
NNTP-Posting-Host: yoda.fdt.net
Mime-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
X-Sender: fubar@yoda.fdt.net
In-Reply-To: <54306t$lh9@news.xs4all.nl>
On 16 Oct 1996, Cor Bosman wrote:
> Dont know if linux can do it, but linux should be easy to patch.
There are actually 2 different patches for linux floating around. I'm
using one that makes no_atime a mount option.
------------------------------------------------------------------
Jon Lewis <jlewis@fdt.net> | Unsolicited commercial e-mail will
Network Administrator | be proof-read for $199/hr.
________Finger jlewis@inorganic5.fdt.net for PGP public key_______
Message-ID: <961016233412.AA12355@dojo>
Date: Wed, 16 Oct 1996 23:34:12 GMT
Path: news.panix.com!panix!feed1.news.erols.com!howland.erols.net!www.nntp.prime
net.com!nntp.primenet.com!nntp.coast.net!dojo!not-for-PROFS
Newsgroups: news.software.nntp
From: Mike O'Connor <mjo@dojo.mi.org>
Subject: Re: atime updates on news spool disk - how about Linux?
References: <jdvicggo2t.fsf@terminus.cs.umb.edu> <53tus7$1p9@nexp.crl.com> <slrn
569r1a.dl.jason@pris.oit.co.uk> <54306t$lh9@news.xs4all.nl>
Reply-To: Mike O'Connor <mjo@dojo.mi.org>
Organization: Sixth Congregational Church of Rodney
Lines: 9
:>Is this supported under Linux? Sounds like a great way of squeezing a bit
:>more performance out of the systems...
Clayton O'Neill's patches for Linux are on http://www.news.erols.com/
I use them. I endorse them. Thanks again!
-- Michael J. O'Connor mjo@dojo.mi.org "Never never never never never never NEVER trust a tiger." -CalvinPath: news.panix.com!panix!feed1.news.erols.com!howland.erols.net!news-peer.gsl. net!news.gsl.net!newspump.sol.net!posts.execpc.com!earth.execpc.com!not-for-mail From: jgreco@ns.sol.net (Joe Greco) Newsgroups: news.software.nntp Subject: Re: atime updates on news spool disk - Sun OS 4.1.4?? Date: 18 Oct 1996 13:12:58 -0500 Organization: sol.net Network Services - Milwaukee, WI Lines: 44 Sender: joegreco@earth.execpc.com Distribution: inet Message-ID: <548hba$oa5@earth.alpha.net> References: <jdvicggo2t.fsf@terminus.cs.umb.edu> <53tus7$1p9@nexp.crl.com> NNTP-Posting-Host: earth-le1.execpc.com
In article <53tus7$1p9@nexp.crl.com>, Eric Mehlhaff <mehlhaff@crl4.crl.com> wrote: >Actually, I'm curious what OS's support this feature, and how it's implemented >or activated on them.
I believe that I started this whole thing back in June or July by hacking it into FreeBSD and then asking if any real FS hackers had a cleaner way to do it..
Clayton O'Neill apparently got wind of this and appears to have written a Linux ext2fs feature implementation to do this. I have seen the patch and could dig it up if needed, but I do not run Linux so I can not say how well it worked, or how to use it.
David Greenman wrote and mentioned that he had a similar FreeBSD patch "almost" written. As mine was rather naive, I chose to trust that someone as familiar with BSD internals as David Greenman would write a much better patch. Both patches allowed a "per filesystem" mount flag to be set to specify "noatime". David's work is in 2.2-CURRENT, but he provided the essentials to make it work on 2.1.5R. I reformatted it a bit to be easier to apply, and have had no problems with it on either 2.1.5R or 2.1.0R.
I will gladly mail it to anyone who wants it along with a short set of instructions.
The NetApp NFS servers appear to support it.
Sun does not appear to support it.
It is generally not a "rocket science" thing to support, so if you don't have it, bug your vendor! It is ideal for Web stuff and news stuff.
>In article <jdvicggo2t.fsf@terminus.cs.umb.edu>, >Elizabeth Schwartz <betsys@cs.umb.edu> wrote: >>Removing these has been mentionned as a way to speed performance. Is there >>anyway to do this under Sun OS 4.1.4? >> >>Any other OS-Specific performance tips would be GREATLY appreciated! >>Especially on speeding up expire...
You can speed up expire (generally) by adding RAM. Also make sure you are using the "delayrm" option, for optimal performance.
... JG
"From " coneill@premier.net Wed Sep 4 23:43:25 1996 Date: Sun, 2 Jun 1996 09:00:59 -0500 (CDT) From: coneill@premier.net To: server-linux@netspace.org, linux-kernel@vger.rutgers.edu Subject: Ext2 speedup (No atime update)
This is an implementation of an idea that I believe was originally mentioned on the FreeBSD hackers mailing list.
The basic idea behind this is that for some applications, there is litle or no point in updating the access time on the file. My primary use for this is on our news server's spool and history disks. These drives are primarily read-intensive, with most of the files being 2-3k. As you can understand, updating a 1k inode every time you read a 2k file isn't really very nice if you don't have to. My tests and examination of the news programs I'm using indicate that it isn't used for anything.
This patch adds two mount-time options, "no_atime" and "update_atime". update_atime is the default and will update atime's normally. no_atime will never update atime's on files, directories, and symlinks. update_atime primarily exists for remounting the filesystem.
This patch has been in production here for almost a week now with no problems. It's very straightforward, and should have little or no side-effects. However, I am wondering, would this be more useful/appropriate as an inheritable ext2 attribute?
Anyway, please send any feedback you have to me, I'd like to hear from anyone that tries it, whether or works or doesn't :)
diff -u -r linux.orig/fs/ext2/dir.c linux/fs/ext2/dir.c --- linux.orig/fs/ext2/dir.c Fri Dec 22 05:00:19 1995 +++ linux/fs/ext2/dir.c Thu May 30 03:04:18 1996 @@ -206,7 +206,7 @@ offset = 0; brelse (bh); } - if (!IS_RDONLY(inode)) { + if (!IS_RDONLY(inode) && !test_opt(sb,NO_ATIME)) { inode->i_atime = CURRENT_TIME; inode->i_dirt = 1; } diff -u -r linux.orig/fs/ext2/file.c linux/fs/ext2/file.c diff -u -r linux.orig/fs/ext2/super.c linux/fs/ext2/super.c --- linux.orig/fs/ext2/super.c Sat May 4 02:07:07 1996 +++ linux/fs/ext2/super.c Thu May 30 15:39:58 1996 @@ -252,6 +252,11 @@ value); return 0; } + } else if (!strcmp (this_char, "no_atime")) { + set_opt (*mount_options, NO_ATIME); + } + else if (!strcmp (this_char, "update_atime")) { + clear_opt (*mount_options, NO_ATIME); } /* Silently ignore the quota options */ else if (!strcmp (this_char, "grpquota") diff -u -r linux.orig/fs/ext2/symlink.c linux/fs/ext2/symlink.c --- linux.orig/fs/ext2/symlink.c Fri Apr 12 02:33:26 1996 +++ linux/fs/ext2/symlink.c Thu May 30 15:42:14 1996 @@ -86,7 +86,7 @@ link = bh->b_data; } else link = (char *) inode->u.ext2_i.i_data; - if (!IS_RDONLY(inode)) { + if (!IS_RDONLY(inode) && !test_opt(inode->i_sb,NO_ATIME)) { inode->i_atime = CURRENT_TIME; inode->i_dirt = 1; } @@ -127,7 +127,7 @@ i++; put_user (c, buffer++); } - if (!IS_RDONLY(inode)) { + if (!IS_RDONLY(inode) && !test_opt(inode->i_sb,NO_ATIME)) { inode->i_atime = CURRENT_TIME; inode->i_dirt = 1; } diff -u -r linux.orig/include/linux/ext2_fs.h linux/include/linux/ext2_fs.h --- linux.orig/include/linux/ext2_fs.h Mon May 20 12:33:56 1996 +++ linux/include/linux/ext2_fs.h Thu May 30 03:01:55 1996 @@ -296,6 +296,7 @@ #define EXT2_MOUNT_ERRORS_RO 0x0020 /* Remount fs ro on errors */ #define EXT2_MOUNT_ERRORS_PANIC 0x0040 /* Panic on errors */ #define EXT2_MOUNT_MINIX_DF 0x0080 /* Mimics the Minix statfs */ +#define EXT2_MOUNT_NO_ATIME 0x0100 /* Don't update the atime */ #define clear_opt(o, opt) o &= ~EXT2_MOUNT_##opt #define set_opt(o, opt) o |= EXT2_MOUNT_##opt diff -u -r linux.orig/mm/filemap.c linux/mm/filemap.c --- linux.orig/mm/filemap.c Sun May 19 15:37:07 1996 +++ linux/mm/filemap.c Thu May 30 21:35:58 1996 @@ -719,7 +719,12 @@ filp->f_reada = 1; if (page_cache) free_page(page_cache); - if (!IS_RDONLY(inode)) { + if (inode->i_sb->s_magic==EXT2_SUPER_MAGIC) { + if (!IS_RDONLY(inode) && !test_opt(inode->i_sb, NO_ATIME)) { + inode->i_atime = CURRENT_TIME; + inode->i_dirt = 1; + } + } else if (!IS_RDONLY(inode)) { inode->i_atime = CURRENT_TIME; inode->i_dirt = 1; } @@ -1097,7 +1102,12 @@ return -EACCES; if (!inode->i_op || !inode->i_op->readpage) return -ENOEXEC; - if (!IS_RDONLY(inode)) { + if (inode->i_sb->s_magic==EXT2_SUPER_MAGIC) { + if (!IS_RDONLY(inode) && !test_opt(inode->i_sb, NO_ATIME)) { + inode->i_atime = CURRENT_TIME; + inode->i_dirt = 1; + } + } else if (!IS_RDONLY(inode)) { inode->i_atime = CURRENT_TIME; inode->i_dirt = 1; }
------------------------------------------------------------------------- Clayton O'Neill Business--> Premier One Inc whatever@oneill.net clayton@premier.net http://www.oneill.net <--Personal http://www.premier.net