[Elphel-support] datarate issue - HDD recording, while streaming.

Andrey Filippov support-list at support.elphel.com
Tue Oct 11 21:37:51 PDT 2011


Nathan,

The intended solution would be to tell streamer minimal size of the camogm
buffer to maintain. And make the streamer "nice" - when the streamer is
awaken by the driver (it happens at each frame, if needed - can sleep for
several frames or until frame #xxx is ready, but it would not take much of
the CPU for the streamer to just check "how is it going" at each frame and
go back to sleep if needed.

At each frame, streamer can read  the buffer size for camogm  -- driver will
tell that (if camogm is set to update "global read pointer"), compare to the
preset minimum and either stream the current frame or go back to sleep and
skip it.

All the data structures (accessible over mmap) are already initialized in
the streamer. I think there needs to be just 10 to 20 (with passing the new
parameter - minimal buffer size) additional lines of code - the most time
consuming would be to get inside the code and find the right place - it was
written/modified by several people at different time (even I touched it
once, but was not the last) so it may not be extremely straightforward. But
if somebody will start working on it - I'll try to help/answer the
questions.

Andrey


On Tue, Oct 11, 2011 at 10:24 PM, Nathan Clark <nathan at nathanclark.com.au>wrote:

> Forwarding a short discussion regarding datarate limitations recording to
> HDD while running streamer simultaneously. Continuing discussion here so
> that others may benefit.
>
> *Problem: *
> Cannot record to HDD with high datarates when streamer is running.
>
> *Cause:*
> Streamer running with high datarate results in high CPU usage causing
> bottleneck.
>
> *Fix:*
> • Reduce CPU requirements of webserver
> • Reduce CPU requirements of streamer
>
> *Is it possible to have the streamer skip every n frames?*
> This would potentially halve the datarate (if skipping every 2nd frame) and
> as such significantly lower the cpu requirement.
> The reduced temporal resolution of the stream would still be okay for
> viewing in many circumstances.
>
> Oleg,
>
> • I get a similar cpu% usage for php instances.
> • I do need the use of the streamer, while recording to SSD over sata.
>
> Cheers,
>
> Nathan
>
> ---------- Forwarded message ----------
> From: Oleg K Dzhimiev
>
> Also, can we move the discussion to our mailing list.
> Thanks
>
> On 11 October 2011 12:41, Oleg K Dzhimiev wrote:
>
>> Nathan,
>>
>> What's the CPU usage for php while recording? I got ~(24+27)% from two php
>> instances. That might be causing the problem - too many php calls from the
>> GUI.
>>
>> In your application do you use the streamer while recording?
>>
>> Best regards,
>> Oleg
>>
>>
>> On 11 October 2011 05:55, Nathan Clark wrote:
>>
>>> As Sebastian advised, I tested the CPU with streamer enabled...
>>> We have a problem!
>>>
>>> CPU usage for streamer is up at 67%
>>> (at 1920x1088, rgb, 25fps, quality 90 - data rate 6.9MB/s)
>>>
>>> The CPU usage directly correlates with data rate, reducing quality to 20
>>> (0.86MB/s) results in streamer CPU usage between 3-9%
>>>
>>> It looks like the streamer is causing this recording issue by
>>> bottlenecking the CPU and thus causing the buffer to overrun...
>>>
>>> But how to circumvent this issue?
>>>
>>>
>>> On 11/10/2011, at 21:23, Sebastian Pichelhofer wrote:
>>>
>>> > Hi Oleg, I suggested we continue this discussion via email.
>>> >
>>> > Below is Nathans original PM to me:
>>> >
>>> >
>>> >
>>> > Hey Sebastian,
>>> >
>>> > As time escapes me, its getting closer and closer to Winnie shooting
>>> > her experimental film for her stereoscopic PHD research...
>>> > I am not ready :|
>>> >
>>> > With problems with my lenses and this sata datarate pain, it's
>>> > becoming quite scary :shock:
>>> >
>>> > But there is always times of stress with all projects! And ultimately,
>>> > I just remind myself how amazing what we have created really is! :D
>>> >
>>> > ----------------
>>> >
>>> > Did you have time to verify the external HDD recording datarate limits
>>> > we talked about some time ago?
>>> >
>>> > Regards Sebastian
>>>
>>>
> _______________________________________________
> Support-list mailing list
> Support-list at support.elphel.com
> http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://support.elphel.com/pipermail/support-list_support.elphel.com/attachments/20111011/bb24f722/attachment-0002.html>


More information about the Support-list mailing list