"Albert D. Cahalan" wrote:
>
> I hope you actually wrote serious proc-using code. If not, you should
> not argue. I wrote the new ps, and I think /proc is a crawling horror.
I've written a fair bit of code to tune clusters and the problems weren't
caused by so much by /proc values being in ASCII, as by the
- lack of regularity in file formats
- inconsistent use of "tag: data", lists of values, and files containing
only one value.
- no standard for repeated values, eg: multiple CPUs
- no standard for nil values (some variables even overloading the zero value,
so there is way to create an average for that variable).
- unnecessary differences across architectures
- inconsistent marking of decimal and hex values, and using
leading-zero padding (which is interpreted as octal by many
parsers)
- lack of discipline and consistency in file and tag naming conventions
- lack of a (user mode) data dictionary containing meta-data such as
types, ranges and meanings of values. Without it, you can't reliabily
plot arbitrary performance variables (is the value continuous or
discrete?) or even create percentages.
A lot of the problems you describe in implmenting "ps" are due to these
factors rather than the data being presented in ASCII format. Moving to
a binary format forces some of the issues to be fixed but doesn't
cure the majority of the list. It may even make some problmes worse
(particularly the nil value problem).
Performance in my code was limited by the inability to grab a whole lot of
related variables at once[1]. I'd be interested in the profile of the "ps"
code: is most of the time spent parsing ASCII, or stuffing about opening and
closing a large number of files and coping with the inconsistent formats?
The trend towards to "one file per variable" is worrying in terms of having
to open a forest of files to get a snapshot of the machine performance,
regardless of whether that file contains ASCII or binary values.
On the other use of /proc, using a binary format for setting and querying
configuration variables would be more inconvenient than setting them
through text. We'd see a proliferation of utilties that do nothing
more than the present "echo 0 > /proc/sys/something". (These C utilities
would also be slower, but that's hardly an advantage as configuration
programs are run too infrequently for that argument to fly. [2])
> My fix would be to add the exact same binary files that Solaris has
> and to make all non-process information invisible.
Would this imply that "ps" and other performance utilities would have to
carefully track the kernel sub-version? This could be a bit painful in
a cluster, especially if the purpose of clustering is high uptime (eg: you'd
be faced with having to reboot all the machines if a security fix came
out for the procps package).
> Real code is C anyway.
Unless you really care about performance. Then you'll find the
tools are written in prototyping langauges so that you can hack
together something to monitor a particular aspect of performance
without the monitoring code taking all year to write.
For this use, a textual /proc is a huge advance over a binary one.
The ease of use aspect also has me concerned, as I've seen a lot
of improvements come about from people "noticing" odd behaviour
and quickly having that confirmed by looking at /proc. I wonder
if a binary /proc would limit the opportunity for this.
> NO. The kernel API is not supposed to be "convenient" for humans.
Unfortunately, filesystems *are* meant to be convenient for humans,
so /proc is at the juncture of opposing expectations.
What's certain at the moment is that the current /proc implementation
isn't convenient for humans *or* programs to parse.
Regards,
Glen
[1] sysctl() doesn't help too much here. What you normally
want to get is the same value for all instances (say of
a CPU or disk). sysctl() is optimised to return at lot
of values for a single instance.
[2] A C (or any) program invoked from the shell is likely to take
more resources than executing a built-in shell statement.
Especially once the likelyhood of I/O is factored in.
-- Glen Turner Network Engineer (08) 8303 3936 Australian Academic and Research Network glen.turner@aarnet.edu.au http://www.aarnet.edu.au/ -- Earth is a single point of failure- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Tue Feb 29 2000 - 21:00:09 EST