ALT Linux Team development discussions
 help / color / mirror / Atom feed
* [devel] Fwd: lj_udrepper: Do you still think the LSB has some value?
@ 2005-09-19 22:24 Alexey Tourbin
  2005-09-19 22:29 ` Dmitry V. Levin
  0 siblings, 1 reply; 41+ messages in thread
From: Alexey Tourbin @ 2005-09-19 22:24 UTC (permalink / raw)
  To: devel

[-- Attachment #1: Type: text/plain, Size: 8322 bytes --]

----- Forwarded message from rss2mail2@solemn -----

From: rss2mail2@solemn
Subject: lj_udrepper: Do you still think the LSB has some value?

Do you still think the LSB has some value? at 17-09-2005 21:33:58
http://www.livejournal.com/users/udrepper/8511.html

   There are still people out there who think that the LSB has any value.
   This just means they buy into the advertisement of the people who have
   monetary benefits from the existence of the "specification", they
   don't do any research, and they generally don't understand ABI issues.

   Just look at the recent LSBv3 "certification" process. Our management
   got pressured by certain parties into declaring that once again we go
   through the process. The v3 spec was extended significantly, some new
   tests were added. And of course, the tests are run against the current
   code base and using the machines people nowadays use. What is the
   result of all this: many many reported bugs.

   This is nothing new, it has been the case for every test run after an
   update of the test suite. And the analysis of the failures is also
   always the same: the bugs are not in the tested code, they are in the
   test suite. There might be occasionally a problem in the code, I think
   I've seen one or two of these, but it's safe to say 90+% of the
   reported bugs are actually problems in the test suite. Look at the LSB
   bug database and all the reported problems if you doubt that.

   Now, considering this, there are some interesting observations to be
   made:

     * 
       The tests certainly at some point worked. I don't doubt that. But
       the kind of errors showed that the author not only doesn't really
       understand the standards, but also cannot really program and
       doesn't understand hardware. This applies also to the code which
       is written by the presumed "professionals" paid by the OpenGroup
       to write tests. Want an example?
       [0] Look at this. This is no isolated incident, I've found this
       kind of problems on many occasions.

     * 
       Some distributions still somehow manage to pass the test suits of
       a new version of the spec. And all this without the people
       reporting any problems and requesting waiving the test. When we
       analyze the code and see that it cannot work we file bugs and wait
       of the test to be waived. If this is not what others do, how can
       they pass the tests? I hope nobody actually cheats, although I'm
       not completely dismissing this. This leaves one solution: create a
       special LSB environment in which the test succeeds. This is
       possible since the geniuses who came up with the whole idea think
       that an own dynamic linker is OK; the latter can then use
       different DSOs which effectively implements a completely separated
       environment. This environment then can be modified to accommodate
       the wrong tests.

       But what does this mean for the user of LSB who is falling victim
       to the illusion of LSB as a stable environment? It means that
       these programs have to work within an environment which a) is
       different from the standard runtime environment and b) is wrong.
       Since the tests are wrong the semantics is not required (after the
       waiver) from other LSB-compliant implementations and therefore the
       adjusted LSB environment differs from those environments which
       didn't change and rely on the waiver. The result of all this is
       that you can have a program which is certified for LSBv3 which
       doesn't run on all LSBv3 certified systems, depending on whether
       the LSB environment worked around the broken test or not. The only
       way to ensure that this specific kind of problem never happens is
       to invalidate every certified distribution once a new waiver is
       released.

     * 
       Another nice things we came across during the LSBv3 testing are
       numerous timing problems. The bug referenced above is also
       depending on timing, which is why it hasn't been discovered
       earlier (the test code is pretty old). When the problem was
       reported the answer we got from the LSB working group was along
       the lines "use a slow uni-processor machine, it is known to work
       there". And what do you know, look at the [1] filing for
       compliance from a certain other distribution. Pay attention to the
       hardware specification. As a colleague of mine correctly said:

         I'm so glad we worry ourselves and put resources into
         complying with
         the industry standard for doorstops.

       What is the value of such a certification? What assurance does
       this give you? Is "don't use fast SMP machines" an acceptable
       answer in any universe, _especially_ when it comes to thread
       tests?

   I think it's time to stop kidding ourselves. It is not possible to
   achieve the goal of 100% binary compatibility except when the same
   binaries are used everywhere. There are test suite bugs (worked around
   or not-yet-discovered), there are whole in the test suite as far as
   specified behavior is concerned (especially when it comes to
   implementation-defined behavior, since this is nothing the
   OpenGroup-provided test suites can test for), and there is the huge
   source of problems called unspecified behavior. Nothing is guaranteed
   when interfaces are used in a way which is defined as unspecified. So
   no test of the _implementation_ can ever help. The applications are
   the issue and here testing is almost entirely lacking. Yes, there is
   an attempt by the LSB to cover this. But it is completely inadequate
   as is and who knows anybody who went through the process or even is
   willing to, considering the extra amount of work this would mean?

   Jim Zemlin and Art Tyde actually came for a visit to talk about these
   issues not too long ago. They were asking for more time as they
   reorganize the LSB (some personnel affects have been visible,
   already). But I think all this is futile, regardless of how much time
   is sunk into this, there will always be holes big enough to drive a
   truck through. And who is writing the tests? Remember, the guy who
   wrote the thread test suite for the OpenGroup is supposed to be an
   expert. Look at the code!

   My advise: but the losses. Remove any claim that the LSB will ensure
   any additional level of assurance for developers. To some extend, I
   think, the claims a scaled back meanwhile, if I understood Art
   correctly. It might be useful to still provide the test suites but
   given their quality, this is questionable at best, too. I'd rather see
   new tests to be written, maybe as an extension of the [2] POSIX test
   suite Intel started. After they added the 100+ reports I sent and
   those others sent the test suite is a somewhat good reflection of who
   a _Linux_ implementation should behave (important: I wrote "Linux" and
   not "POSIX").

   Until the LSB loses the monetary backing this is unlikely to happen,
   though. The main reason: people who deal with standards
   professionally. They of course have everything to lose. These are the
   same people who brought you useless crap like the Linux ISO spec
   (based on some old LSB version). These people are paid to participate
   in calls, meetings, get there travels to exotic places financed. Did
   you look at the list of meeting places for ISO meetings? I hope the
   penny squeezers in the companies financing these standardization
   groups (not only the LSB, there are many more) realize the waste of
   money most of these efforts are and introduce more control. Yes,
   something like the Austin Group working group is useful, this is an _API_
   standard (as opposed to _ABI_ in case of LSB) which makes it much more
   realistic and useful. But ISO Linux? Shudder...

   [0] http://bugs.linuxbase.org/show_bug.cgi?id=1055
   [1] http://www.opengroup.org/lsb/cert/display_product.tpl?CALLER=cert_prodlist.tpl&_pr_id=564
   [2] http://posixtest.sourceforge.net/


----- End forwarded message -----

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 41+ messages in thread
end of thread, other threads:[~2005-10-04 14:34 UTC | newest]

Thread overview: 41+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2005-09-19 22:24 [devel] Fwd: lj_udrepper: Do you still think the LSB has some value? Alexey Tourbin
2005-09-19 22:29 ` Dmitry V. Levin
2005-09-19 23:14   ` [devel] " Alexey Tourbin
2005-09-20 12:21     ` Dmitry V. Levin
2005-09-20 18:55       ` Alexey Tourbin
2005-09-21  6:48         ` Kirill A.Shutemov
2005-09-21  8:45           ` Ivan Fedorov
2005-09-21  9:05             ` Kirill A.Shutemov
2005-09-21 10:23               ` Vitaly Ostanin
2005-09-21 10:44                 ` Kirill A.Shutemov
2005-09-21 13:06               ` Денис Смирнов
2005-09-21 13:25                 ` Alexey Tourbin
2005-09-21 13:27                 ` Kirill A.Shutemov
2005-09-21 13:37                   ` [devel] [JT] system languages Michael Shigorin
2005-09-21 13:47                   ` [devel] Re: Fwd: lj_udrepper: Do you still think the LSB has some value? Grigory Batalov
2005-09-21 22:18                   ` Денис Смирнов
2005-09-22  9:08                     ` Alexey Tourbin
2005-09-22  9:13                       ` Денис Смирнов
2005-09-22  9:59                         ` [devel] [JT] " Alexey Rusakov
2005-09-22 10:16                           ` Денис Смирнов
2005-09-22 10:22                             ` [devel] " Sergey Bolshakov
2005-09-22 10:24                               ` Alexey Rusakov
2005-09-22 12:53                                 ` Vital Khilko
2005-09-22 13:38                                   ` Alexey Rusakov
2005-09-23 17:29                                 ` Денис Смирнов
2005-09-23 17:29                               ` Денис Смирнов
2005-09-24 10:32                                 ` Sergey Bolshakov
2005-09-24 15:31                                   ` [devel] " Mikhail Zabaluev
2005-10-04 14:03                               ` [devel] " Stanislav Ievlev
2005-10-04 14:34                               ` Sergey V Turchin
2005-10-03 14:34                       ` [devel] " Maxim Tyurin
2005-09-21  9:09           ` [devel] miniruby Michael Shigorin
2005-09-21  9:10             ` Kirill A.Shutemov
2005-09-21  9:46               ` [devel] miniruby Michael Shigorin
2005-09-21  7:01         ` [devel] [JT] OCaml (was: Do you still think... ) Grigory Batalov
2005-09-28  6:39           ` [devel] Re: [JT] OCaml docs Grigory Batalov
2005-09-28  7:55             ` Alexey Tourbin
2005-09-28  8:13               ` Vitaly Ostanin
2005-09-28  9:02                 ` Alexey Tourbin
2005-09-28 10:32                   ` Vitaly Ostanin
2005-10-03 14:37               ` Maxim Tyurin

ALT Linux Team development discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://lore.altlinux.org/devel/0 devel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 devel devel/ http://lore.altlinux.org/devel \
		devel@altlinux.org devel@altlinux.ru devel@lists.altlinux.org devel@lists.altlinux.ru devel@linux.iplabs.ru mandrake-russian@linuxteam.iplabs.ru sisyphus@linuxteam.iplabs.ru
	public-inbox-index devel

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://lore.altlinux.org/org.altlinux.lists.devel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git