Lighting Sphere - A New Controllable Lighting System for Multi-view Lighting

Lighting Sphere: A New Controllable Lighting System for Multi-view Lighting

The lighting sphere is an effective solution to the problem of high precision human pose estimation, which requires an efficient fit of natural light to human pose in large scenes. lighting sphere is an efficient solution to this task: it provides illumination by programmatically activating and deactivating a number of LEDs arranged in the sphere while capturing a photo. We aim to design a lighting sphere, which uses multiple sets of lighting units to illuminate, through the master control unit for each lighting unit control, to achieve multi-angle, multi-view and high-speed lighting control problems, so that the required lighting object to provide various angles of lighting conditions to meet the shooting conditions and lighting tasks. For the generated results will be portrait deduction and three-dimensional portrait rendering, so as to visualize the lighting effect of each angle and analyze the lighting performance.

This is my undergraduate project, all completed by me

1. Introduction

How to capture high quality and realistic 3D models of the human body has been a central problem in the computer vision and machine learning community. Although we have witnessed the birth of several advanced human capture systems such as 8i, Omnivor, Intel, etc. in recent years, they are still somewhat difficult to capture high frequency details and they can only simulate fixed lighting conditions. There has been a lot of work on rendering 3D human body using algorithms for natural light at all angles, and for machine learning for light rendering requires a large number of datasets for training, and the acquisition of datasets is still a huge challenge, and it would be labor intensive to use existing sunlight for dataset capture, so it is very important to design a system that can simulate natural light at all angles to replace this process. It is important to design a system that can simulate natural light at all angles to replace this process.

Our lighting sphere built on HAND-3D-STUDIO. All lighting units are controlled by a control board, powered by a power conversion board, visualized and controlled by the lighting sphere control software, and the final effect is observed by a 3D portrait rendering method. The final lighting sphere is controlled in real time by 54 lighting units to achieve a multi-angle controlled light effect, with different lighting angles and the number of lighting units equal to 54. Each lighting unit can provide up to 2000hz PWM dimming frequency, achieving up to 100 levels of multi-level adjustment effect.

2. Method

Our system has three parts in total, including: the hardware lighting system control board, the hardware lighting system lighting unit, and the 3D portrait rendering method.

Hardware lighting sphere lighting unit

The lighting unit controls six high-power white LED beads of up to 1W to achieve adjustable brightness. Although the brightness of the LED beads can be adjusted by adjusting the current, the Pulse Width Modulation (PWM) pulse width modulation is more suitable for the design of the lighting unit. If PWM is used to control the input voltage of the LED, when the PWM period is reduced to a certain degree, the strobe will no longer be visible, but will become a change in the size of the luminosity under the human eye, so as to achieve LED brightness adjustment.

LED lighting requires a low-voltage DC constant-current source to work, while the common power supply is usually 220V AC power supply, which can not be used directly for LED power supply. Therefore, the 220V AC power supply needs to be converted to a low-voltage constant-voltage DC power supply. Our light unit contains about 340 LEDs, each with a power of about 1W and a total power of about 340W. In order to meet the need, a low-voltage DC constant voltage source with a total power of about 340W needs to be used, and considering the impact of conversion efficiency, an appropriate power margin needs to be selected to ensure the stable operation of the power supply, so a computer host power supply with an output power of 500W is selected to ensure its stable power supply, and an output voltage of 12V is selected to meet the required power.

Hardware lighting sphere control board

The control board outputs 15 PWM signals per MCU by arraying 12 STM32F103C8T6s, and outputs the 180 PWM signals used to control the light unit through the signal enhancement and through the network cable POE method. And the control signals of the control board are controlled by obtaining real-time control data from the upper computer software through RS485 communication.

After the PWM signal is sent from the MCU, it needs to go through the 74HC04 inverter to reverse the signal first, in order to prevent the sudden high level at the moment of power-on, after which it needs to go through the ULN2003 as power amplification, and finally output through the net port. Pull-up and pull-down resistors are added at both ends of the ULN2003 Darlington array to increase the driving current and improve the circuit power.

3D portrait rendering method

Metashape is a 3D modeling approach based on RGB images, which is based on multi-view geometric composition and image processing techniques that generate 3D models of objects using computer algorithms by inputting multiple sets of photographs of the target object captured at various angles. In Metashape, key point matching is used to match feature points from multiple photographs and calculate the geometric relationships between them, such as the camera’s pose (position and orientation) and the location of 3D points. This information can be further used to generate 3D models, point clouds, textures, etc.

3. Experiment

The lighting sphere control board is implemented through an array of 12 MCUs, which can control up to 60 lighting units by enhancing the PWM signal and outputting it through the network port. The lighting sphere control board can be controlled in real time by a PC-based visualization software and set specific operation modes for automatic control. The lighting sphere can achieve multi-angle, multi-viewpoint and high-speed lighting control issues, so that the required lighting objects to provide various angles of light conditions to meet the shooting conditions and lighting tasks.

Lighting sphere through the real-time control of 54 lighting units to achieve a multi-angle controlled light effect, the different light angle and the number of light units equivalent to 54. Each lighting unit can provide up to 2000hz PWM dimming frequency, to achieve up to 100 levels of multi-level adjustment effect.

Its use of HDRI full-view steel ball can be observed from multiple angles of the lighting sphere, 54 which can successfully achieve the multi-angle lighting effect on the need to light the target, through the upper computer program for real-time and efficient visual control.

The system can light the object in real time from 18 angles, and the lighting of the top lighting unit can be observed from the bottom of the system, and this test was achieved in the angle switching mode with an interval of 300ms.

If all 54 light units are turned on, a circular light area will be formed, with the brightness set to 3 and the camera sensitivity set to 3200. The light source from all directions is perpendicular to the subject’s face, and the distribution of shadows and highlights is even, making the appearance of the portrait look more uniform, and there is almost no light source biased to one side, only the lower part due to the absence of light units, making the facial features on this side more prominent, but making the portrait more three-dimensional.

If the 54-bit light unit open vertical column single side, will form a single side of the angle light source, its brightness is set to 10, other light units open brightness of 3 to fill the light, the camera sensitivity is set to 1600, the light source from the direction perpendicular to the subject’s face, shadows and highlights in the side more prominent, making the appearance of the portrait looks more three-dimensional prominent, other direction light source to ensure the overall brightness, can successfully achieve angle light lighting.

After three-dimensional reconstruction through metashape, you can achieve a good single-side light lighting effect, if the original image will be directly reconstructed, the effect will be very poor, basically can not see the human form, when the use of MODNet keyed portrait, the effect becomes better as shown below, the direction of the light source and the subject’s face perpendicular, shadows and highlights on the side more prominent, making the appearance of the portrait looks more three-dimensional prominent, other directional light source to ensure the overall brightness, can successfully achieve angle light lighting.

For the front of the portrait of a single side light lighting effect, the direction of the light source is perpendicular to the subject’s face, open the front of the subject’s light unit and set the brightness to 8, the other light unit brightness is set to 3 for fill light, the camera sensitivity is set to 1600, shadows and highlights in the front more prominent, but the two sides of the unlit part of the three-dimensional shadow, making the appearance of the portrait looks more three-dimensional prominent, other directional light source to ensure the overall brightness, can successfully achieve angle light lighting.

4. Conclusion

The lighting sphere is a multi-view controllable lighting system control board with an array of 12 MCUs, which can be enhanced with PWM signals and output through a network port to control up to 60 lighting units. The lighting sphere control board can be controlled in real time by the PC-based visualization software, and set specific operation modes for automatic control. The lighting sphere can achieve multi-angle, multi-viewpoint and high-speed lighting control issues, so that the required lighting objects to provide various angles of light conditions to meet the shooting conditions and lighting tasks.

Lighting sphere through the real-time control of 54 lighting units to achieve a multi-angle controlled light effect, the different light angle and the number of light units equivalent to 54. Each lighting unit can provide up to 2000hz PWM dimming frequency, achieving up to 100 levels of multi-level adjustment effect.

Video

This video is an actual demonstration of the Lighting sphere