[Elphel-support] Cameras and processing modules for Self Driving Cars Vision System

[Marius Slavescu]

Thanks a lot Andrey!

This is exactly what I was looking for as a base platform for OSSDC 
Smart Camera.

I'm away for the weekend, I'll get back to you with more details on Monday.

Thank you,
Marius


On 06/30/2017 11:14 AM, Andrey Filippov wrote:
> Hello Marius,
>
> Elphel cameras through the history used multiple different sensors 
> from different manufacturers, and the cameras are designed to be 
> flexible and being able to accommodate new sensors as well. Currently 
> off the shelf we have 5 Mpix sensors that we consider to be a good 
> balance between sensitivity and resolution - OnSemi/Aptina MT9P006 
> with 2.2umx2.2um pixels as well as (still only partially supported by 
> the software drivers) smaller pixel 14MPix. For the book scanning 
> project we used huge 35mm format Kodak CCDs connected to the same 
> system boards. At that time our system board 10313, but all the later 
> ones (10333, 10353 and the current 10393) are backward compatible - 
> just some software for the older sensors is missing.
>
> All our code is under GNU GPLv3+, excluding Linux kernel drivers that 
> have to be GNU GPLv2(+), hardware designs are under CERN OHL. 
> Mechanical parts are documented on Elphel Wiki, most parts are also 
> viewable with x3d using our FreeCAD-based converter from STEP files 
> libraries into x3d assemblies , like here: 
> https://www.elphel.com/wiki/Elphel_camera_assemblies .  All FPGA code 
> is GPLv3.
>
> We support FSF and Richard Stallman in not using term "IP", so we do 
> not have or develop any of it. But we have both plenty of the Verilog 
> code in our git repository (all our code is there without any 
> exclusions), the main one is 
> https://git.elphel.com/Elphel/x393/tree/dct (there is a block diagram 
> if you scroll down the page). This code is really free software, it 
> does not rely on any "black boxes" or "IP" and can be completely 
> simulated with Icarus Verilog (GNU GPLv2)
>
> For FPGA development we also provide advanced tools based on Eclipse 
> IDE that support Xilinx Vivado (and older ISE) , Intel/Altera Quartus 
> and can accommodate other architectures. It Integrates modified 
> version of Veditor, Icarus Verilog and Cocotb (for simulation with 
> Python - 
> https://blog.elphel.com/2016/07/i-will-not-have-to-learn-systemverilog/ 
> ) and interacts with Xilinx proprietary tools over ssh/rsync :
> https://git.elphel.com/Elphel/vdt-plugin
> https://blog.elphel.com/2016/05/tutorial-02-eclipse-based-fpga-development-environment-for-elphel-cameras/
> https://www.youtube.com/watch?v=9g8udf_nhZE
>
> 3d camera - no it is not https://www3.elphel.com/stereo_setup , it is 
> something much more advanced, based on both the new hardware and the 
> new software and is targeted to do most of the job directly in the 
> camera FPGA. I hope to  post some demo links to the intermediate 
> results soon.
>
> And self driving cars is one of the target applications for this 
> project. Mars rovers can probably benefit from it too - long rage 
> passive 3d reconstruction requires less valuable power than lasers.
>
> Andrey
>
>
>
>
>
>
>
>
>     Hi Andrey,
>
>     Thank you for getting back to me!
>
>     I'm trying to figure out what FPGAs and components I should get to
>     build smart cameras that supports 1-4 x 1080P 60FPS (or better)
>     low light (2+um pixel) sensors, for open source robotics and self
>     driving car development.
>
>     I understand these solutions may not be cheap or easy to approach,
>     I'm trying to see what can be used best and relatively future
>     proof as starting point for open source development in this area.
>
>     OSSDC being a global community driven project, I'm also looking at
>     affordable solutions that I can promote to be used for development
>     and testing of SDC algorithms all around the world, as OSSDC
>     target is global full autonomy (not only applicable in certain areas).
>
>     Would be great if you could give me some ideas of what
>     configurations I should try (my budget is very limited for now),
>     and which IPs may help get started faster.
>
>     I'll post soon more details about the OSSDC platform architecture
>     and Smart Camera, here are some things we collected, more
>     discussions happed on OSSDC Slack (ossdc.org):
>
>     https://github.com/OSSDC/OSSDC-SmartCamera/issues/1
>
>     https://medium.com/@mslavescu/what-about-putting-a-computer-vision-processor-on-the-camera-or-sensor-platform-itself-d0622b24f5c
>
>     I'm also looking into HDMI capture solutions (for both cameras and
>     gaming consoles/PCs):
>
>     https://medium.com/@mslavescu/hdmi-capture-and-analysis-on-fpga-for-just-150-8b34652b0b0c
>
>     https://medium.com/@mslavescu/get-ready-to-race-ai-with-us-at-ossdc-org-b741e266e362
>
>     I assume this is the 3D camera you mention, would be possible to
>     use a different sensor to increase the frame rate and also is it
>     possible to get uncompressed video to the PC (to do further
>     processing on raw data using Nvidia GPUs):
>
>     https://www3.elphel.com/stereo_setup
>
>     Stereo (multi) camera with dense depth maps would perfect for SDC,
>     that what I think is necessary to achieve full autonomy at
>     relatively low cost.
>
>     Thank you,
>     Marius
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://support.elphel.com/pipermail/support-list_support.elphel.com/attachments/20170701/290f4852/attachment.html>