Re: [malware-list] scanner interface proposal was: [TALPA] Intro to a linux interface for on access scanning

[Peter Dolding]

On Tue, Aug 19, 2008 at 9:07 AM, Eric Paris <eparis@xxxxxxxxxx> wrote:
> On Tue, 2008-08-19 at 00:40 +0200, Pavel Machek wrote:
>> Hi!
>>
>> > > On Mon, Aug 18, 2008 at 02:15:24PM +0100, tvrtko.ursulin@xxxxxxxxxx
>> > wrote:
>> > > > Then there is still a question of who allows some binary to declare
>> > itself
>> > > > exempt. If that decision was a mistake, or it gets compromised
>> > security
>> > > > will be off. A very powerful mechanism which must not be easily
>> > > > accessible.  With a good cache your worries go away even without a
>> > scheme
>> > > > like this.
>> > >
>> > > I have one word for you --- bittorrent.  If you are downloading a very
>> > > large torrent (say approximately a gigabyte), and it contains many
>> > > pdf's that are say a few megabytes a piece, and things are coming in
>> > > tribbles, having either a indexing scanner or an AV scanner wake up
>> > > and rescan the file from scratch each time a tiny piece of the pdf
>> > > comes in is going to eat your machine alive....
>> >
>> > Huh? I was never advocating re-scan after each modification and I even
>> > explicitly said it does not make sense for AV not only for performance but
>> > because it will be useless most of the time. I thought sending out
>> > modified notification on close makes sense because it is a natural point,
>> > unless someone is trying to subvert which is out of scope. Other
>> > have
>>
>> Why do you think non-malicious applications won't write after close /
>> keep file open forever?
>
> If you ask this one more time without reading the many times I've
> answered these questions I think I'm going to explode.
>
> Permissions checks are done on open/read.  Decisions are invalidated at
> mtime update, which INCLUDES mmap after close!  I don't care if you keep
> your file open forever, if you wrote to it, we are just going to scan it
> next time a process decided to open/read it.  Please stop confusing this
> already long and almost pointless thread with implementation details
> that have repeatedly been explained.
>
You will see latter where what you just said fails and its issue is
preventable too downloader with build in previewer.

Funny enough solution to this is fairly simple.   But does require
looking at a white list methods and LSM.

Two major ways.    White list format check method tells you that file
is not complete enough so black list scanning is not required yet.  Ok
lighter than running 5000 black list signatures over it each time a
new block gets added.

Unfortunately scan of all pieces as them come in for possible threat
still has to be done reason on videos and the like people in some
applications start playing them before the download is complete.
Lots of video and audio formats have blocks that can be cleared piece
by piece.   Just like a bittorrent can scan and pass block by block so
does a white list scanner need to be able to.  More creative use can
be void block insert into some formats.   Ie part downloaded block at
X replaced with a equal to blank block to the player  that is external
to the download tool.   This is damaged data access prevention.
Nicely prevents some stability issues and gives users extra features.

White list scanner knowing the format can detect when enough segments
are in a file to run a black list scan avoiding jack hammering the cpu
eating black list scan.

Dealing with bittorrent clients with built in preview is a pain in the
you know what.   Since are they reading the file to send to someone
else are they reading the file to display in there internal viewer or
do they take straight from there download buffer to internal view.
Even worse lots of bittorrent streams are encrypted and cannot be
scanned while network packets.   So second solution required a LSM
around the downloader preventing it in case of breach being able to go
anywhere in the system.   LSM only allows access to files that the
downloader has downloaded by other applications with more rights when
its pasted White list and needed black list scanning.

Getting this to work without using white list of known format method
and LSM is basically imposable because a black list is going to take
far to much cpu time scanning incomplete files.

This is one of the reasons why the system is 4 tech groups.  HIDS,
white list methods, black list methods and LSM.      Without all 4 you
are stuffed trying to make a prevention system work because you will
get snagged somewhere trying to cover all setups and events.   A
combination is required some setups can get away without the black
list.   There are many points where current black list methods are
flawed or hurt way too much.

Some of these will require looking at two extra tech groups.   1 being
intergation into applications.   2 providing apis for applications to
do particular protocols and blocking out direct network access other
than threw those apis.

Lot of windows anti-virus people are way too focused on black list.
White list might annoy you from time to time but it can also grant
features that users may not want to give up.

Peter Dolding
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/