From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on sa.int.altlinux.org X-Spam-Level: X-Spam-Status: No, score=-1.3 required=5.0 tests=AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 Date: Wed, 12 May 2010 15:58:02 +0300 From: Michael Shigorin To: devel@lists.altlinux.org Message-ID: <20100512125801.GQ24342@osdn.org.ua> Mail-Followup-To: devel@lists.altlinux.org Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.4.2.1i Subject: [devel] Fwd: Native upstart scripts X-BeenThere: devel@lists.altlinux.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: ALT Linux Team development discussions List-Id: ALT Linux Team development discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 May 2010 12:58:06 -0000 Archived-At: List-Archive: List-Post: úÄÒÁ×ÓÔ×ÕÊÔÅ. ÷ÄÒÕÇ ÐÒÉÇÏÄÉÔÓÑ... ÉÚ ÏÔËÒÙÔÙÈ ÉÓÔÏÞÎÉËÏ×... É ×Ó£ ÔÁËÏÅ. ----- Forwarded message from Jacek Konieczny ----- Date: Tue, 4 May 2010 11:17:41 +0200 From: Jacek Konieczny To: PLD Devel Subject: Native upstart scripts Hello, In PLD Th, we have upstart as the /sbin/init daemon for some time, but it is only used to start the old 'SysVinit' scripts from /etc/rc.d/init.d. To make full use of Upstart features, like process supervising, parallel start, etc. we should find a way to include Upstart support in our packages. Though, we should not replace the current, working solution. I think Upstart support should be implemented as an option, coexisting with current solution, so the administrator may choose what he prefers and even use init.d for some services and upstart for other. My proposition: - packages will provide upstart configuration files in /etc/rc.d/upstart - those could be linked or copied to /etc/init/subsys when needed - chkconfig would link/unlink the files when requested (global configuration option and command line option to prefer upstart) - user could copy them manually and modify them if needed - rc-scripts won't start/stop services via /etc/rc.d/init.d scripts when /etc/init/subsys/$name exists - 'service' script would use 'initctl' instead of /etc/rc.d/init.d when /etc/init/subsys/$name exists - scripts in /etc/rc.d/init.d would emit 'started' and 'stopped' events when necessary, so upstart services can rely on that What do you think? Should I try to prepare a proof-of-concept implementation? P.S. Does anybody read those 'long' posts I send to pld-devel-en? ;) _______________________________________________ pld-devel-en mailing list pld-devel-en/lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en ----- End forwarded message ----- ----- Forwarded message from Jacek Konieczny ----- Date: Tue, 4 May 2010 12:53:40 +0200 From: Jacek Konieczny To: pld-devel-en/lists.pld-linux.org Subject: Re: Native upstart scripts On Tue, May 04, 2010 at 12:04:39PM +0200, Patryk Zawadzki wrote: > > - scripts in /etc/rc.d/init.d would emit 'started' and 'stopped' > > ševents when necessary, so upstart services can rely on that > > I'd opt for having 2 separate -init subpackages, one with the current > rc.d contents and one with an upstart job description and a simple > rc.d wrapper that runs "start $foo", "stop $foo" etc. There is one more thing why I would prefer to keep both old-style init.d script and the new upstart configuration together: we often have something more than 'start' 'stop' and 'status' implemented in the init scripts. Some of these actions would have to be copied to upstart script somehow (initialization before first run), other probably cannot be handled by upstart means. sshd init script builds host keys the first time it is started. This functionality is available also via '/etc/rc.d/init.d/sshd init' and would have to be copied to the upstart configuration to and maintained in the two places. As starting and stopping of services is quite different in LSB init scripts and upstart and reusing code does not make sens, in other cases, like 'init' the code may be reused and upstart script could just call '/etc/rc.d/init.d/sshd init'. Also, there are things like '/etc/rc.d/init.d/lighttpd configtest' which are not part of normal job control and thus not applicable to upstart config, but still useful. Greets, Jacek ----- End forwarded message ----- ----- Forwarded message from Jacek Konieczny ----- Date: Tue, 4 May 2010 19:51:49 +0200 From: Jacek Konieczny To: pld-devel-en/lists.pld-linux.org Subject: Re: Native upstart scripts On Tue, May 04, 2010 at 07:40:43PM +0300, Elan Ruusam??e wrote: > On Tuesday 04 May 2010 12:17:41 Jacek Konieczny wrote: > > My proposition: > > - packages will provide upstart configuration files in /etc/rc.d/upstart > > - those could be linked or copied to /etc/init/subsys when needed > > š - chkconfig would link/unlink the files when requested (global > > š š configuration option and command line option to prefer upstart) > > š - user could copy them manually and modify them if needed > > i don't like the idea that the links are managed via some script, i'd like > easily to boot to upstart-mode or sysvinit-mode with a kernel commandline, or > something in /etc/sysconfig/system the both solutions should be available and > configured at the same time. I'll take this into account. Though keeping both configured could cause mistakes like starting a service again the different way. I guess this can be solved somehow anyway > also i'd not invent new paths, but use /etc/init for upstart scripts. Recent upstart allows hierarchical names. I thought a separate namespace (subsys/* or rc/*) for the rc-scripts equivalents would be a good idea. Now I think it will be probably better to use subdirs for local stuff (/etc/init/local -> local/* jobs) and sub-tasks (e.g. /etc/init/postgresql.conf ??? 'postgresql' task to start/stop whole postgresql and /etc/init/postgresql/instance.conf ??? 'postgresql/instance' describing each instance) instead. Greets, Jacek ----- End forwarded message ----- ----- Forwarded message from Jacek Konieczny ----- Date: Wed, 5 May 2010 12:34:34 +0200 From: Jacek Konieczny To: pld-devel-en/lists.pld-linux.org Subject: Re: Native upstart scripts On Tue, May 04, 2010 at 07:40:43PM +0300, Elan Ruusam?e wrote: > i don't like the idea that the links are managed via some script, i'd like > easily to boot to upstart-mode or sysvinit-mode with a kernel commandline, or > something in /etc/sysconfig/system the both solutions should be available and > configured at the same time. I thought about that and cannot see how it could be implemented. When there are upstart configs for some services then Upstart will start them by the dependencies inside and I cannot see a way to stop this from a kernel command line and still have the flexibility of which services should be handled by Upstart and which the old way. Greets, Jacek ----- End forwarded message ----- ----- Forwarded message from Jacek Konieczny ----- Date: Wed, 5 May 2010 12:31:55 +0200 From: Jacek Konieczny To: pld-devel-en/lists.pld-linux.org Subject: Re: Native upstart scripts On Wed, May 05, 2010 at 11:20:30AM +0200, Patryk Zawadzki wrote: > On Tue, May 4, 2010 at 11:17 AM, Jacek Konieczny wrote: > > In PLD Th, we have upstart as the /sbin/init daemon for some time, but > > it is only used to start the old 'SysVinit' scripts from > > /etc/rc.d/init.d. To make full use of Upstart features, like process > > supervising, parallel start, etc. we should find a way to include > > Upstart support in our packages. Though, we should not replace the > > current, working solution. > > BTW: http://0pointer.de/blog/projects/systemd.html Great. This fixes some problems of Upstart??? but??? This would look great to me if I haven't seen several other ???great SysVinit replacements???. All of them were much better than SysVinit and even had some ???working implementations???. Often long before the Upstart came up. And everybody have been still using SysVinit. Upstart is the only one which caught on. It is terrible, with its big incompatibilities between each version??? poor documentation (at least on the web), impractical event system. Nevertheless it is still much better than SysVinit with the pile of shell scripts starting daemons doing anything not to be managed (double-forking, etc.). Unfortunately /sbin/init is so critical, that we cannot even experiment much, especially with something no one else uses. And it is hard to have multiple /sbin/init implementation side by side in one distribution. They all differ too much not only in the configuration file formats but in the whole philosophy of the configuration (not only ???how to describe a service configuration??? but even ???what is a service configuration???). One could think of making some kind of an abstraction layer, like our rc-inetd, but that is a very bad idea. We would probably lose most of the advantages of each init implementation then. We should rather think how can we provided a few very different configuration items, each for a different init implementation, with one package. I am losing my enthusiasm about implementing the ???native upstart support??? in PLD when I think systemd may eventually mature enough and we would need to start from the beginning??? And I need a good init system now. Currently I use SysVinit + daemontools in my PLD-based project. It generally works, but is not elegant nor optimal. Even upstart, when properly deployed, would be better and systemd could be much better, but I guess it is a matter of future. Pozdrowienia, Jacek ----- End forwarded message ----- ----- Forwarded message from Jacek Konieczny ----- Date: Wed, 5 May 2010 14:32:19 +0200 From: Jacek Konieczny To: pld-devel-en/lists.pld-linux.org Subject: Re: Native upstart scripts On Wed, May 05, 2010 at 02:13:51PM +0200, Patryk Zawadzki wrote: > On Wed, May 5, 2010 at 2:03 PM, ??ukasz Jerna?? wrote: > > Well, after seeing how many people have weird problems with PulseAudio > > I would be cautious about using another thing šthought out by > > Lennart... > > To be fair you have to admit that a vast number of the problems > experienced while using pulseaudio are actually bugs in kernel audio > modules and/or alsa API abuse (think opal, ekiga etc.). I would rather say it is because of putting another audio server where it is not needed at all. ALSA alone does its job well enough nowadays (and if it does not, it should be fixed not wrapped with another layer). Fortunately things have not gone too far yet and a system without PulseAudio can still be set up (there was no such freedom in the dark ages of ESD and ARTS). This /sbin/init things are not that easy. We will always need some init daemon??? though I still won't chose an implementation only basing on the fact that the idea is great. It must work and be maintained (or at least stable as SysVinit) too. Greets, Jacek ----- End forwarded message ----- ----- Forwarded message from Jacek Konieczny ----- Date: Fri, 7 May 2010 15:33:21 +0200 From: Jacek Konieczny To: PLD Devel Subject: Native upstart scripts ??? implemented Hello, Your volunteer has done his job :) I have made the proof-of-concept implementation of native upstart job control. I did it a bit different way than I initially suggested ??? taking into account your comments and improving my own ideas by trial and errors. My main requirements were fulfilled: - this doesn't break current /etc/rc.d/init.d/* scripts - this can coexist with the current /etc/rc.d/init.d/* script based job control - /etc/init.d/* scripts are quite LSB compatible even when the jobs are controlled by upstart. Currently all the changes are applied to 'upstart_native' branch, so the solution may be tested before merging into the distribution. As it doesn't seem to break anything I guess it can be merged very soon (after the weekend?) unless someone shows me some big problem with the code or has a good idea on how some things could be done better. I have modified: - rc-scripts to provide facilities needed for upstart-controlled services - syslog-ng as an example of a generic service implementation other services rely on ??? how to make job depend on a 'syslog' service not a specific 'syslog-ng' implementation - openssh-server as an example of some system service relying on network and syslog - postgresql as an example of multi-instance service. It also shows how to run a service with non-root uid (it is strange upstart doesn't provide this function directly) To play with that build rc-scripts, syslog-ng, openssh and postgresql from the 'upstart_native' branch, upgrade to these builds (this should change nothing in how your system works) and then install syslog-ng-upstart, openssh-upstart and postgresql-upstart. When the *-upstart packages are installed you can still control the services with /sbin/service or /etc/rc.d/init.d/$service scripts or directly with the 'initctl' tool. The way it is implemented gives two options of booting the PLD system: 1. the traditional ??? serialized, slow but verbose and colorful way ??? just do not install the *-upstart packaged or boot with 'pld.no-upstart' 2. the new event-based, parallelized, auto-respawning way, but less verbose - instal the *-upstart packages and let them do their job Some documentation for the rc-scripts+upstart usage is here: http://svn.pld-linux.org/cgi-bin/viewsvn/rc-scripts/branches/upstart_native/doc/upstart.txt?rev=11395&view=markup Patryk Zawadzki wrote: > I'd opt for having 2 separate -init subpackages, one with the current > rc.d contents and one with an upstart job description and a simple > rc.d wrapper that runs "start $foo", "stop $foo" etc. -upstart subpackages done. Addin "-init" makes no sense, as current upstart job handling implementaion relies on the init.d scripts for LSB compatibility and doing things not doable with bare Upstart (non-SIGHUP-reloading, 'checkconfig', advanced status monitoring). Elan Ruusam??e wrote: > i don't like the idea that the links are managed via some script, i'd like > easily to boot to upstart-mode or sysvinit-mode with a kernel commandline, or > something in /etc/sysconfig/system the both solutions should be available and > configured at the same time. Done. No chkconfig patching, no script-managed-links, only a minor update do /sbin/service added. The new, upstart event-based boot may be disabled by a kernel command line option: "pld.no-upstart". > also i'd not invent new paths, but use /etc/init for upstart scripts. Done. But this has some consequences: - The package-provided /etc/init/*.conf files are config files. Expect user to modify them in any way. Do not put things, that may change on every upgrade, there. - Package-provided job description could conflict with used locally or provided by third-party packages. I suggest using /etc/init/local/*.conf for local jobs and keeping /etc/init/*.conf files for /etc/rc.d/init.d/* equivalents (put extra jobs needed for the functionality into a subdirectory) Greets, Jacek ----- End forwarded message ----- ----- Forwarded message from Jacek Konieczny ----- Date: Sun, 9 May 2010 08:57:57 +0200 From: Jacek Konieczny To: pld-devel-en/lists.pld-linux.org Subject: Re: Native upstart scripts ??? implemented On Sun, May 09, 2010 at 02:52:57AM +0300, Elan Ruusam?e wrote: > i'd rather avoid completely the new subpackage here, if needed move > the /etc/init dir to filesystem package to avoid dirdeps pulling upstart, and > use conflicts tag for the current requires tag. Patrys asked for subpackages and it makes sense - the easiest to choose between upstart-way and the-old-way per package. > syslog-ng/syslog-ng.init: > ... > upstart_controlled --except configtest > > i don't really understand how flush-logs is handled, It is not. 'service syslog-ng flush-logs' will tell you that. On the other hand - with upstart 'reload' does the same. Of course, flush-logs can be implemented in the init script. I guess it should if it is used by anything else (logrotate?) > also how do you specify multiple excludes remains unclear. When '--except' is used, then all what follows are the excludes. > also, configtest before reload/restart action would be really important to > have in upstart as well, considering that we restart services on rpm > upgrades. does upstart have such concept after all as restart/reload in > scripts? No. 'reload' in upstart is 'kill -HUP', anything else must be re-implemented in the init script. Restart is 'stop; start'. Hmmm... Now I can see 'restart with configtest' may be easily implemented. Whenever '--except configtest' is used, 'upstart_controlled' can call configtest before trying restart. configtest on start makes little sense with upstart IMHO. > ps: would be nice to know where's source of documentation, for example i did > not find myself description of job file directives. >>From the rc-scripts init.txt: The syntax of the ``/etc/init/*.conf`` files is described in the init(5) man page. And yes, looking for a current Upstart documentation in the web gives no good results. Greets, Jacek ----- End forwarded message ----- ----- Forwarded message from Jacek Konieczny ----- Date: Sun, 9 May 2010 11:00:57 +0200 From: Jacek Konieczny To: pld-devel-en/lists.pld-linux.org Subject: Re: Native upstart scripts ??? implemented On Sun, May 09, 2010 at 11:39:14AM +0300, Elan Ruusam??e wrote: > On Friday 07 May 2010 16:33:21 Jacek Konieczny wrote: > > Hello, > > > > Your volunteer has done his job :) > > seems there's some deadlock with initctl emiting Isn't that another result of https://bugs.launchpad.net/upstart/+bug/406397 > also seems the "nice service name" is lost there (see sshd part). How is that supposed to work and where is that 'sshd part'? > also seems there's no our typical restart service after package upgrade, > if you've upgrading upstart-enabled service. > %define upstart_post() \ > if [ -f /var/lock/subsys/"%1" ] ; then \ > /sbin/service --no-upstart "%1" stop \ > /sbin/service "%1" start \ > fi The service will be restarted after main package upgrade. Though we have the real problem of -upstart subpackage here: the restart should probably be done when both main and -upstart packages are upgraded. > anyway, ps of stall: > root 25974 0.4 0.5 15596 6064 pts/3 S+ 11:23 0:00 \_ poldek -u upstart --up --sn carme openssh-server-upstart syslog-ng-upstart > root 27078 0.4 0.5 12516 5804 pts/3 S+ 11:24 0:00 \_ > rpm --upgrade -vh --root / /var/cache/poldek/http_carme.pld-linux.org..glen.th.i686/syslog-ng-3.0.5-2.1.i > root 27181 0.0 0.0 1872 584 pts/3 S+ 11:24 0:00 \_ /bin/sh /home/users/glen/tmp/rpm-tmp.59975 2 > root 27184 0.0 0.0 1872 604 pts/3 S+ 11:24 0:00 \_ /bin/sh /sbin/service syslog-ng restart > root 27187 0.0 0.0 2000 808 pts/3 S+ 11:24 0:00 \_ /bin/sh /etc/rc.d/init.d/syslog-ng restart > root 27321 0.0 0.0 5716 972 pts/3 S+ 11:24 0:00 \_ /sbin/initctl emit started JOB=syslog-ng SERVICE=syslog I guess --no-wait for emit should be used here. Though there is something blocking ??? emit waits for some action on the 'started' event to finish and some service starting on this even has probably locked up (most probably due to https://bugs.launchpad.net/upstart/+bug/406397 and broken /etc/init/*.conf file (I have recently fixed those for openssh and syslog-ng)). Greets, Jacek ----- End forwarded message ----- ----- Forwarded message from Jacek Konieczny ----- Date: Sat, 8 May 2010 22:25:27 +0200 From: Jacek Konieczny To: pld-devel-en/lists.pld-linux.org Subject: Re: Native upstart scripts On Sat, May 08, 2010 at 10:41:07PM +0300, Elan Ruusam?e wrote: > On Tuesday 04 May 2010 12:17:41 Jacek Konieczny wrote: > > What do you think? Should I try to prepare a proof-of-concept > > implementation? > > any plans of moving rc.sysinit also to upstart? No. I don't think we would gain a lot. Reimplementing all these stuff via Upstart keeping all this working may be quite difficult and maintaining both will be a hell. And no, switching fully to Upstart at this point is not a good idea. Have anybody recently upgraded upstart 0.5 to 0.6 on a production machine? Have anybody used 'expect daemon' instead of 'expect fork' or made a similar mistake? Any of these would break such system so it won't even shut down or reboot properly. Upstart is great in many points much much better than SysVinit. But it is not ready to be the only option for a distribution. Let's add upstart support to more and more packages, but do not make it a critical component of a PLD system. Greets, Jacek _______________________________________________ pld-devel-en mailing list pld-devel-en/lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en ----- End forwarded message ----- -- ---- WBR, Michael Shigorin ------ Linux.Kiev http://www.linux.kiev.ua/