[Elphel-support] datarate issue - HDD recording, while streaming.

[Sebastian Pichelhofer]

I ran a full measurement cycle.

With and without streamer enabled, I also reduced the AJAX call
frequency in camogmgui to once every 5 seconds to not give PHP an
additional CPU load in the camera. All values are averaged over 5
seconds.

JPEG Quality and data-rate are relative not absolute as the data-rate
depends on the actual image content but I kept it as reference during
the tests.

All tests were done with 1920x1088 at 25fps RGB Color mode.

Regards Sebastian

On Mon, Oct 17, 2011 at 18:12, Oleg K Dzhimiev <oleg at elphel.com> wrote:
> Nathan,
> I was able to reduce the PHP's cpu usage and could run recording
> 1920x1088 at 25fps, quality 93, with streamer working, single frame size
> ~300K-~400K,
> streamer ~60% cpu, camogm ~30% cpu (same w/o streamer).
> The problem is that the php calls (status requests) are "too frequent" so
> that they eat ~30%. I used my own interface (originally for our panorama
> camera) for recording where the status calls interval can be programmed.
> So that interval was 5 sec - can be more frequent, 2 sec worked also.
> It could be a solution if Sebastian implements it in his GUI.
> Regards,
> Oleg
>
> On 11 October 2011 22:24, Nathan Clark <nathan at nathanclark.com.au> wrote:
>>
>> Forwarding a short discussion regarding datarate limitations recording to
>> HDD while running streamer simultaneously. Continuing discussion here so
>> that others may benefit.
>> Problem:
>> Cannot record to HDD with high datarates when streamer is running.
>> Cause:
>> Streamer running with high datarate results in high CPU usage causing
>> bottleneck.
>> Fix:
>> • Reduce CPU requirements of webserver
>> • Reduce CPU requirements of streamer
>> Is it possible to have the streamer skip every n frames?
>> This would potentially halve the datarate (if skipping every 2nd frame)
>> and as such significantly lower the cpu requirement.
>> The reduced temporal resolution of the stream would still be okay for
>> viewing in many circumstances.
>> Oleg,
>> • I get a similar cpu% usage for php instances.
>> • I do need the use of the streamer, while recording to SSD over sata.
>>
>> Cheers,
>> Nathan
>> ---------- Forwarded message ----------
>> From: Oleg K Dzhimiev
>>
>> Also, can we move the discussion to our mailing list.
>> Thanks
>> On 11 October 2011 12:41, Oleg K Dzhimiev wrote:
>>>
>>> Nathan,
>>> What's the CPU usage for php while recording? I got ~(24+27)% from two
>>> php instances. That might be causing the problem - too many php calls from
>>> the GUI.
>>> In your application do you use the streamer while recording?
>>> Best regards,
>>> Oleg
>>>
>>> On 11 October 2011 05:55, Nathan Clark wrote:
>>>>
>>>> As Sebastian advised, I tested the CPU with streamer enabled...
>>>> We have a problem!
>>>>
>>>> CPU usage for streamer is up at 67%
>>>> (at 1920x1088, rgb, 25fps, quality 90 - data rate 6.9MB/s)
>>>>
>>>> The CPU usage directly correlates with data rate, reducing quality to 20
>>>> (0.86MB/s) results in streamer CPU usage between 3-9%
>>>>
>>>> It looks like the streamer is causing this recording issue by
>>>> bottlenecking the CPU and thus causing the buffer to overrun...
>>>>
>>>> But how to circumvent this issue?
>>>>
>>>>
>>>> On 11/10/2011, at 21:23, Sebastian Pichelhofer wrote:
>>>>
>>>> > Hi Oleg, I suggested we continue this discussion via email.
>>>> >
>>>> > Below is Nathans original PM to me:
>>>> >
>>>> >
>>>> >
>>>> > Hey Sebastian,
>>>> >
>>>> > As time escapes me, its getting closer and closer to Winnie shooting
>>>> > her experimental film for her stereoscopic PHD research...
>>>> > I am not ready :|
>>>> >
>>>> > With problems with my lenses and this sata datarate pain, it's
>>>> > becoming quite scary :shock:
>>>> >
>>>> > But there is always times of stress with all projects! And ultimately,
>>>> > I just remind myself how amazing what we have created really is! :D
>>>> >
>>>> > ----------------
>>>> >
>>>> > Did you have time to verify the external HDD recording datarate limits
>>>> > we talked about some time ago?
>>>> >
>>>> > Regards Sebastian
>>>>
>>
>> _______________________________________________
>> Support-list mailing list
>> Support-list at support.elphel.com
>> http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com
>>
>
>
> _______________________________________________
> Support-list mailing list
> Support-list at support.elphel.com
> http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Datarate Measurements.pdf
Type: application/pdf
Size: 24882 bytes
Desc: not available
URL: <http://support.elphel.com/pipermail/support-list_support.elphel.com/attachments/20111017/40efc5e9/attachment.pdf>