From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Sat, 12 Dec 2009 06:51:10 +0300 From: Alexey Tourbin To: ALT Linux Team development discussions Message-ID: <20091212035110.GE9864@altlinux.org> Mail-Followup-To: ALT Linux Team development discussions References: <4B223383.5050703@mmedia2.kemsu.ru> <20091211182800.GC9864@altlinux.org> <20091211195602.GV13584@osdn.org.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="BZaMRJmqxGScZ8Mx" Content-Disposition: inline In-Reply-To: <20091211195602.GV13584@osdn.org.ua> Subject: Re: [devel] parallel builds, again (was: [Fwd: [#16528] COMPLETE (try 23)]) X-BeenThere: devel@lists.altlinux.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: ALT Linux Team development discussions List-Id: ALT Linux Team development discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2009 03:51:47 -0000 Archived-At: List-Archive: List-Post: --BZaMRJmqxGScZ8Mx Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 11, 2009 at 09:56:02PM +0200, Michael Shigorin wrote: > On Fri, Dec 11, 2009 at 09:28:00PM +0300, Alexey Tourbin wrote: > > > Does at@ or ldv@ have any ideas how to further improve > > > builder performance? There never was so stressfull load on > > > infrastructure and hope never be in near future, but will we > > > wait so long time if it happens again? > > First off (perhaps in case you didn't notice), we're doing > > blazing fast. >=20 > I think it's just as "fast" as when one stands for an hour > trying to book tickets for the train departing roughly in an > hour, while the operator can do nothing but wait until the lock > somewhere is released. So you argue "blazing fast" isn't fast enough. Well, you see, this whole business of building packages actually can't work miracles. Realistically, we're doing pretty well. That's all I have to say. > Single-threaded build is getting us nowhere when CPU clocks are > effectively frozen at <=3D 3GHz while cores are plenty; ARM case > is even more sensitive to single system performance being a > bottleneck (as discussed on devel-ports@). So what are you trying to say. More cores can work more miracles. Nope. > > As a matter of fact, a task with 1000 of packages, running for > > the second time, can complete within only hours. If you still > > think this isn't fast enough, consider that 1000 packages > > actually are 1/n-th part of the whole repo. >=20 > Well you have done great job at shaving seconds off one CPU core > but there are at least a few more sitting around idle, no? > If you still think it's fast enough, consider that it could be > 2x or 4x faster. I think it can't. > > > Why packages without direct python dependencies (runtime or > > > buildtime) was blocked by python task? > > Because what we do is "purely functional" builds, with no > > exceptions. >=20 > ...which is pretty expensive as already discussed -- and for > quite a few cases could be replaced by opportunistic approach > using cached previous package build results in hope that things > won't shift much, and falling back to "strict" model in case they > actually did. Man, do you really want to even ENGAGE in all this stuff? It is far more complicated than... oh well... --BZaMRJmqxGScZ8Mx Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEARECAAYFAksjEy4ACgkQfBKgtDjnu0b4IQCfY3k6GuJTK7Zvzll9H+lBITgj /skAn3Ipw14OkS8GXfLYDRW6+aCjyUNf =qKQb -----END PGP SIGNATURE----- --BZaMRJmqxGScZ8Mx--