Apache2.0下的vodd的内存占用大问题

提示1

Ah. My mistake... I thought brigade structures were malloc'd. If they're
from a pool, then we have problems (see below). Also, presuming that we make
them mallocs, then we need to be proactive (see below).
我的错。我认为brigade结构被分配过的。如果从内存池里分配，那么我们会有问题（参看如下内容）。同样，假定我们使他们分配，那么我们需要来提前处理（参看如下）。
On Sun, Dec 31, 2000 at 03:21:43PM -0800, rbb[at]covalent.net wrote:
>...
> We should not be trying to outsmart our allocation system. Brigades are
> incredibly small, specifically so that they can be allocated and just left
> for the pools to deal with. We are literally taking about a total of
> three pointers.
我们不应该尝试来过度考虑分配系统。Brigade难以想象地小，具体到可以被分配和只让内存池来管理。我们只是书面上管理三个指针的总数。
Three pointers multiplied by the content-size of the response is too big.
Heck, one byte in proportion to the content-size is too big.

Like I said: if a brigade is created in proportion to the content-size, then
it must be destroyed.
如果brigade以部分内容创建，那么它必须销毁。
If the brigade is created as a fixed count per response, then yes: leave it
for the pools.
如果brigade为每个回应以固定个数创建，留给内存池处理。
But that fixed vs proportional is important... proportional a Bad Thing(tm).

> We just don't need the added function calls associated
> with destroying a bucket_brigade. This requires that we kill a cleanup
> that isn't going to do anything anyway. The memory doesn't actually get
> freed when we destory the brigade, and if the brigade has just be
> concated, then it is going to be empty.
>
> Basically, what you are asking for, is to do always kill a cleanup that is
> basically a no-op whenever we destroy a brigade. There is no reason to do
> this. We don't free any memory, because the memory is allocated out of a
> pool, and we never free memory from pools. If we call destroy_brigade
> whenever we concat, all we are doing is searching a linked list and
> calling a function that is essentially a no-op because the brigade is
> empty.

Yes, the brigade will be empty, so no worries in terms of buckets.

I think the problem is that the brigade itself is allocated from a pool. As
you say: that means we can't toss it after a concat. It would seem that we
should reconsider allocating brigades from pools.

[. specifically: consider that ap_r* creates a new brigade each time they are
called. this means the number of brigades is proportional to the content
size. we must be proactive in clearing them out ]

In summary:

1) don't use pool allocations for brigade structures. Use malloc and a
tricky cleanup (we can't use a dumb cleanup because then we're doing a
pool-alloc (for the cleanup info) for every brigade creation again).
不要为brigade结构使用内存池分配器。使用malloc和一个机警的cleanup（我们不能使用一个伪cleanup，因为我们不是一个内存池分配（为了cleanup信息））。
2) call ap_brigade_destroy() when we're done with a brigade
当我们使用完了brigade后，调用ap_brigade_destroy.
[. we don't need to be as careful with destroying brigades that are fixed in
count. the pool cleanup will toss those. ]

Cheers,
-g

提示2

Buckets live forever until you destroy them.

Brigades live as long as the pool they're in (normally r->pool) and
automatically delete any buckets still in them when that pool cleans up

The bucket allocator will allocate as much memory as is needed for the
maximum amount of simultaneously-alive buckets *ever*. c->bucket_alloc
is a pointer to an allocator that may live as long as the whole thread.

In other words, the easiest ways to leak memory with buckets are to take a
bunch of buckets out of their brigade and then forget about them, or leave
them in the brigade and then forget about the brigade, in which case
they'll at least get cleaned up when r->pool goes away, but that still
might result in a lot of buckets simultaneously allocated before the
request is done. Or you could have a filter that buffers way more stuff
than it should, but that's usually a more obvious bug

提示3

Just brain-dumping, the design issue here is, per Jeff's analysis and subsequent
discussion:

1. brigades are allocated out of pools
2. every call to apr_brigade_split allocates a new brigade
3. every time a FLUSH bucket it sent down the filter stack, it causes at least
one call to apr_brigade_split

fixes for this could be either:

fix (1), allocate brigades out of the bucket-allocator so that they can really
be free'd (very intrusive since many filters presume brigades are never really
destroyed)

fix (3), adjust all filters to ensure that they don't allocate a number of
brigades per request (and hence, memory allocated) which is proportional to
number of FLUSH buckets sent.