[Date Prev][Date Next]
Re: Re: Transaction logging and OpenLDAP
- To: "Tech Guy" <firstname.lastname@example.org>
- Subject: Re: Re: Transaction logging and OpenLDAP
- From: "matthew sporleder" <email@example.com>
- Date: Fri, 11 Aug 2006 15:25:18 -0400
- Cc: openldap-software@OpenLDAP.org
- Content-disposition: inline
- Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=LPFhd8cCyv0QZimxoYKYIOlyXF5uSuVXOew9O6Ep5fAxayuiB/X4ODYmEtK05uMOFVR3z5miQ9zlz+8h3LnzdRyILJRGp8UiZUP9hL3+1qIYACFXGfoFGrHF0b16mqWbdszIKjBPjkaeqOnQDlFLEjiPyq3cNrsGZgY3CpRutHk=
- In-reply-to: <firstname.lastname@example.org>
- References: <email@example.com> <firstname.lastname@example.org>
On 8/11/06, Tech Guy <email@example.com> wrote:
--- matthew sporleder <firstname.lastname@example.org> wrote:
> * 0 * * * /usr/bin/db_checkpoint -1 -h /data/ldbm
> <-- I hope that's a
> typo. It would create 60 checkpoints at the 0 hour.
Thanks for the link. Yes, that was a typo.
> I thought I remembered reading about a DB_CONFIG
> variable that would,
> essentially, turn of transactions for bdb (thus
> eliminating the logs).
> However, I don't think there's a good reason to do
> so. My testing
> has shown that ldbm (the non-transactional bdb) is
> simply slower. The
> I/O from these logs files is minimal, and they don't
> even take up much
> space with autoremove turned on.
> The DB_LOG_AUTOREMOVE makes the db_archive
> redundant, by the way.
> I don't think forcing the checkpoint is necessary,
Like I mentioned in my post, the DB_LOG_AUTOREMOVE
does not appear to always work in certain stress
tests. Although it greatly helps in reducing the logs
I have seen that after such stress add tests that
running "db_archive -d" alone does not remove the log
files, until I force a checkpoint. I have seen
previous posts with older bdb on long running
transactions that should have been fixed in this
release. It almost appears that BDB believes the log
files (~10 log files on a 20hr run) are still holding
Has anyone else encountered this issue with these
I haven't noticed that, but I use a much smaller log size, which
forces more rotation.
> if you don't want to keep the log files around, and
> only doing one/day
> doesn't make a lot of sense unless it's followed by
> a few other
> scripts for backups. Why not run them every hour?
> Or just set the
> set_lg_max in DB_CONFIG to control the size of the
> transaction logs,
> and autoremove will take care of the ones that
> aren't needed for
> running. 10MB of write activity for an hour isn't
> exactly taxing on
> any modern system.
> I'll note, however, that getting rid of these logs
> eliminates one
> possible avenue of recovery. All backups must now
> be taken using
> ldapsearch (for live backups) or with slapcat (for
> downed backups).
> What problem are you actually trying to solve?
Yes, running the job more frequently is a possibility,
although based on testing, the log files should only
grow to 120Mb in 24 hours. I agree that 10Mb of writes
is not intensive, but due to log rotation the server
can easily run out of space in a few days. The
"set_lg_max" setting in DB_CONFIG is present and
capped at 100Mb. We do have live backups to take care
of catastrophic failures.
100MB seems like a large transaction log file.
If you can get predictable failure of the autoremove not working, I
think it's worthy of a bug report. If you do see it, then I don't
think your cron jobs will hurt anything.