I've not time to write a big long thing out fresh right now (major deadline, ahoy!) but I use a module internally called AntiSpew which might be of interest. I'll paste in the perldoc. The text is bad because it was only intended ffor personal consumption. Sorry. If you can't make head-nor-tail of it, do ask.
NAME
Dan::AntiSpew - A package to help regulate spew
MOTIVATION
This package helps regulate spew. Spew is a problem caused
by overuse of a resource, deliberate or accidental. The
use of the resource consumes a resource which persists for
a considerable time (for example a log-file entry).
Overuse could cause these things to become indavertantly
full. Many of them are only purged irregularly by a human
often, unfortunately, upon seeing an error from a fully-
consumed resource. This module restricts use to a particu-
lar rate and provides a facility to monitor the rate at
which the resource is actually being consumed in compari-
son to this maximum. It places hard constraints on disk
usage whilst allowing for transients and peaks. The rate
and peak-allowances are configurable. The algorithm is
simple to understand and implement, it follows from the
problem and doesn't rely on constructs such as moving
averages or complex heuristics like sepearte peak and con-
tinuous flow accounting which can lead to effects in par-
icular regions.
ALGORITHM
The antispew algorithm is based upon a concept called
atime. atime is never in the past, it is either the pre-
sent or in the future. Two things move on atime. Whenever
atime is the present moment, atime moves forward with cur-
rent time, staying at the present moment. Whenever
resource is consumed, atime moves forward a fixed amount,
known as `n'. Therefore, if atime is some distance in the
future, if the resource is consumed at a rate of `1/n' per
second, it will neither move forward nor backward (over
time) with respect to the present moment. If the resource
is consumed quicker, it will slowly disappear into the
future. If consumed slower, it will move back to the pre-
sent time, and hten stay at that time. A certain region
beyond the present time, `m' seconds, is known as the per-
mitted region for the atime. A request is only allowed if
the atime would remain within the permitted region. There-
fore the atime always does remain in that region. There-
fore if our resource usage is quiescent, we can suddenly
consume `m/n' resources, the peak load. This peak need not
be sudden, alternatively it could simply be a marginally
increased rate until equivalent to `m/n' resources over
the stead-state rate of `1/n'.
The implementation of this algorithm is natural, we store
the last recorded atime on disk. When a resource is
requested we read in that time and set it to the present
if that time is now in the past. We then advance the time
and, if in the permitted region, write that time and allow
the request, othewise leave the atime as it is. The pro-
portion of the peak allowance used is `(a-x)/m' where `x'
is the current wall-time. This is also recorded so that
steady increases in this allowance can suggest to an oper-
ator that more resource is needed to sustain demand a long
itme before the resource is actually exhausted.
The maximum number of more resources consumed in the
future at any time is a contant (m/n) plus the sustained
rate (n) multiplied by the time.
Your proposals make technical sense. I predict, therefore, that a simpler, superficially easier to understand scheme, which deals only with a few common cases and occasionally relies on trust and obscurity, will be adopted.
![[livejournal.com profile]](https://www.dreamwidth.org/img/external/lj-community.gif){kind=small}
![[identity profile]](https://www.dreamwidth.org/img/silk/identity/openid.png){kind=small}
![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
timeasmymeasure
posting juice
Date: 2003-02-17 11:02 am (UTC)NAME Dan::AntiSpew - A package to help regulate spew MOTIVATION This package helps regulate spew. Spew is a problem caused by overuse of a resource, deliberate or accidental. The use of the resource consumes a resource which persists for a considerable time (for example a log-file entry). Overuse could cause these things to become indavertantly full. Many of them are only purged irregularly by a human often, unfortunately, upon seeing an error from a fully- consumed resource. This module restricts use to a particu- lar rate and provides a facility to monitor the rate at which the resource is actually being consumed in compari- son to this maximum. It places hard constraints on disk usage whilst allowing for transients and peaks. The rate and peak-allowances are configurable. The algorithm is simple to understand and implement, it follows from the problem and doesn't rely on constructs such as moving averages or complex heuristics like sepearte peak and con- tinuous flow accounting which can lead to effects in par- icular regions. ALGORITHM The antispew algorithm is based upon a concept called atime. atime is never in the past, it is either the pre- sent or in the future. Two things move on atime. Whenever atime is the present moment, atime moves forward with cur- rent time, staying at the present moment. Whenever resource is consumed, atime moves forward a fixed amount, known as `n'. Therefore, if atime is some distance in the future, if the resource is consumed at a rate of `1/n' per second, it will neither move forward nor backward (over time) with respect to the present moment. If the resource is consumed quicker, it will slowly disappear into the future. If consumed slower, it will move back to the pre- sent time, and hten stay at that time. A certain region beyond the present time, `m' seconds, is known as the per- mitted region for the atime. A request is only allowed if the atime would remain within the permitted region. There- fore the atime always does remain in that region. There- fore if our resource usage is quiescent, we can suddenly consume `m/n' resources, the peak load. This peak need not be sudden, alternatively it could simply be a marginally increased rate until equivalent to `m/n' resources over the stead-state rate of `1/n'. The implementation of this algorithm is natural, we store the last recorded atime on disk. When a resource is requested we read in that time and set it to the present if that time is now in the past. We then advance the time and, if in the permitted region, write that time and allow the request, othewise leave the atime as it is. The pro- portion of the peak allowance used is `(a-x)/m' where `x' is the current wall-time. This is also recorded so that steady increases in this allowance can suggest to an oper- ator that more resource is needed to sustain demand a long itme before the resource is actually exhausted. The maximum number of more resources consumed in the future at any time is a contant (m/n) plus the sustained rate (n) multiplied by the time.Well donw
Date: 2003-02-17 11:15 am (UTC)Pavlos