[Elphel-support] datarate issue - HDD recording, while streaming.

[Nathan Clark]

Forwarding a short discussion regarding datarate limitations recording to
HDD while running streamer simultaneously. Continuing discussion here so
that others may benefit.

*Problem: *
Cannot record to HDD with high datarates when streamer is running.

*Cause:*
Streamer running with high datarate results in high CPU usage causing
bottleneck.

*Fix:*
• Reduce CPU requirements of webserver
• Reduce CPU requirements of streamer

*Is it possible to have the streamer skip every n frames?*
This would potentially halve the datarate (if skipping every 2nd frame) and
as such significantly lower the cpu requirement.
The reduced temporal resolution of the stream would still be okay for
viewing in many circumstances.

Oleg,

• I get a similar cpu% usage for php instances.
• I do need the use of the streamer, while recording to SSD over sata.

Cheers,

Nathan

---------- Forwarded message ----------
From: Oleg K Dzhimiev

Also, can we move the discussion to our mailing list.
Thanks

On 11 October 2011 12:41, Oleg K Dzhimiev wrote:

> Nathan,
>
> What's the CPU usage for php while recording? I got ~(24+27)% from two php
> instances. That might be causing the problem - too many php calls from the
> GUI.
>
> In your application do you use the streamer while recording?
>
> Best regards,
> Oleg
>
>
> On 11 October 2011 05:55, Nathan Clark wrote:
>
>> As Sebastian advised, I tested the CPU with streamer enabled...
>> We have a problem!
>>
>> CPU usage for streamer is up at 67%
>> (at 1920x1088, rgb, 25fps, quality 90 - data rate 6.9MB/s)
>>
>> The CPU usage directly correlates with data rate, reducing quality to 20
>> (0.86MB/s) results in streamer CPU usage between 3-9%
>>
>> It looks like the streamer is causing this recording issue by
>> bottlenecking the CPU and thus causing the buffer to overrun...
>>
>> But how to circumvent this issue?
>>
>>
>> On 11/10/2011, at 21:23, Sebastian Pichelhofer wrote:
>>
>> > Hi Oleg, I suggested we continue this discussion via email.
>> >
>> > Below is Nathans original PM to me:
>> >
>> >
>> >
>> > Hey Sebastian,
>> >
>> > As time escapes me, its getting closer and closer to Winnie shooting
>> > her experimental film for her stereoscopic PHD research...
>> > I am not ready :|
>> >
>> > With problems with my lenses and this sata datarate pain, it's
>> > becoming quite scary :shock:
>> >
>> > But there is always times of stress with all projects! And ultimately,
>> > I just remind myself how amazing what we have created really is! :D
>> >
>> > ----------------
>> >
>> > Did you have time to verify the external HDD recording datarate limits
>> > we talked about some time ago?
>> >
>> > Regards Sebastian
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://support.elphel.com/pipermail/support-list_support.elphel.com/attachments/20111012/f9a5e15f/attachment-0002.html>