Issue
J. Eur. Opt. Society-Rapid Publ.
Volume 19, Number 1, 2023
Advancing Society with Light, a special issue from general congress ICO-25-OWLS-16-Dresden-Germany-2022
Article Number 29
Number of page(s) 7
DOI https://doi.org/10.1051/jeos/2023020
Published online 02 June 2023

© The Author(s), published by EDP Sciences, 2023

Licence Creative CommonsThis is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1 Introduction

Emerging innovations like tactile Internet [1] and mobile healthcare [2] necessitate high-capacity, low latency and secure communication networks [3]. In the past decades, single-mode fiber (SMF)-based communications primarily provided the required backbone networks driving such high performance technologies. In order to overcome capacity exhaustion in SMFs called the nonlinear Shannon limit [4], multiplexing in the spatial domain of FMFs, e.g. implemented by digital signal processing [5], is considered an essential complement to conventional multiplexing techniques [6]. SDM [7, 8] enables to gainfully exploit spatially parallel optical paths in FMFs the design of which can be optimised for such transmission [9]. The interplay between FMF and SDM allows for data rates beyond 1 Pbits−1 in one single waveguide [10].

When launching data to FMF, those spatial modes that overlap with the incident input beam are excited [11]. A straightforward and low-loss multiplexer is provided by a photonic lantern [12]. By adiabatic tapering, a launching waveguide can be physically coupled to the FMF. However, photonic lanterns barely provide selectivity and no adaptability of the launching conditions after fabrication and crosstalk compensation has to be carried out on the receiver-side inflating the digital processing overhead.

In contrast, mode multiplexers based on MPLC [13] provide programmability. This method allows to map several input spots to the modes of the FMF. In a free-space configuration, multiple reflections between a mirror and a phase-modulating element, e.g. an SLM, carry out the desired transformation. Depending on the phase pattern displayed on the SLM, the transformation can be further adapted for instance to implement predistortion techniques to tailor light fields making crosstalk compensation on the receiver-side obsolete [14, 15], or to enhance information security [16, 17].

In recent papers on MPLC [10, 14, 18, 19], the SLM phase masks are calculated offline using the wavefront matching algorithm (WMA) [18, 20]. However, such pre-processing is critical in terms of errors including pixel-precise alignment or incident angle tolerances [21]. Alternatively, machine learning algorithms provide a suitable framework for the spatial mapping of light fields. For example, in NN-based image reconstruction intensity images captured at multimode fiber output are allocated to the corresponding input field [2224]. In turn, with machine learning it is also possible to generate SLM phase masks performing the desired light shaping task. This enables among others real-time wavefront shaping through complex media [25], nonlinear spatiotemporal light control [26], or diffractive optical NN, which are based on similar spatial transformations like those with the MPLC considered [27].

In this work, we present a method to advance the mode shaping procedure by a smart data-driven online-calibration, where the entire MPLC apparatus is considered as black box. By training an NN, we create a digital twin of the setup called Model-NN. With the Model-NN, we gain knowledge about the experimental environment. Afterwards, we train another NN called Actor-NN that controls the model. This idea is inspired by the work published in Ref. [28]. Within our method, we adapt the Actor-Model approach to enhance the MPLC performance for SDM applications. In the following section, we will introduce the MPLC technique in general. Afterwards, we explain the procedure of mimicking the experimental setup as a digital twin. Finally, we present an implementation to shape intensity images of the EMNIST handwritten letters dataset [29].

2 Smart calibration for multiplane light conversion

A general scheme of an MPLC device is shown in Figure 1. An array of incoherent input spots is entering the MPLC apparatus. Input spots with Gaussian profile E in illuminate the SLM at the first reflection passage performing phase-only modulation E out = E in e i ϕ SLM $$ {E}_{\mathrm{out}}={E}_{\mathrm{in}}\cdot {e}^{i{\phi }_{\mathrm{SLM}}} $$(1)applying the spatial phase-term ϕ SLM. Using multiple of those free-beam passages, both amplitude and phase profiles are modulated. As a result, the MPLC output should match the desired output modes, which can be calculated by solving the MAXWELL equations.

thumbnail Figure 1

Optical MPLC setup considered in our work. Two spatially seperated Gaussian input spots are incident to the SLM. After two reflection passages between SLM and mirror (M), the output beams are imaged by a 4f-telescope onto a camera where intensity images are recorded. Both the FMFs facet and the camera are at a distance d ≈ 1.5 cm from the SLM. A smart calibration based on artificial intelligence (AI) is implemented to generate proper SLM phase masks in order to shape desired output modes.

Although the WMA provides a numerically correct solution to calculate the SLM phase masks, it is prone to errors that can barely be considered during offline calculation [21]. Among others, there are delicate misalignment in pixel-precise positioning of the phase patterns, incident angle tolerance and mismatching propagation distances between SLM and mirror that cause crucial performance drops. To compensate for such errors, we propose an intelligent online calibration procedure that considers the entire MPLC device as black box. The anticipated NN structure is shown in Figure 2.

thumbnail Figure 2

Structure of the envisioned NN architecture. (a) Training of a single-layer Model-NN. Training data consists of 2k uniformly distributed random SLM phase masks and the corresponding intensity images measured at the MPLC output. This generates a digital twin of the MPLC setup. (b) Another single-layer called Actor-NN is trained on 2k intensity images according to the EMNIST data set. The Actor-NN is used to predict phase masks, which are the input for the Model-NN. When training the Actor-NN, the Model-NN is fixed. (c) The predicted phase mask of the trained Actor-NN is applied to the SLM and the intensity is measured. We achieve a correlation of Γ = 0.65 compared to the Model-NNs predicted intensity.

In a first step, the MPLC is set up with two passages transfering two input beams. To gain knowledge about the black box, a single-layer NN is trained. SLM phase masks ϕ SLM are used as NN input and the output is the corresponding intensity image I with I = c ϵ 2 E out 2 $$ I=\frac{c\epsilon }{2}\cdot \left\langle {E}_{\mathrm{out}}^2\right\rangle$$(2)is measured by a camera. c denotes the speed of light and ϵ 0 the permittivity of the media.

We use a single-layer perceptron as NN architecture for both, Actor- and Model-NN. The input pixels representing SLM phase masks are thereby connected straightforwardly to the pixels of the output intensity image and vice-versa, resulting in 2.304.000 trainable parameters. The NN and its internal structure is shown in Figure 3. For proper training, 2k uniformly distributed random phase masks and corresponding intensity images are used. For both NNs, we use 1600 samples for training, and 400 for test, respectively. As loss function, we use standard mean-squared error (MSE) with MSE = 1 N - 1 i = 1 N ( x i - y i ) 2 , $$ \mathrm{MSE}=\frac{1}{N-1}\sum_{i=1}^N ({x}_i-{y}_i{)}^2, $$(3)where x i denotes the ground truth and y i the prediction for N samples. Compared to the MSE we use as loss function, the performance of the NN is indicated by the correlation Γ with Γ = i = 1 N ( x i - x ¯ ) ( y i - y ¯ ) i = 1 N ( x i - x ¯ ) 2 i = 1 N ( y i - y ¯ ) 2 . $$ \mathrm{\Gamma }=\frac{\sum_{i=1}^N ({x}_i-\bar{x})({y}_i-\bar{y})}{\sqrt{\sum_{i=1}^N ({x}_i-\bar{x}{)}^2}\sqrt{\sum_{i=1}^N ({y}_i-\bar{y}{)}^2}}. $$(4)

thumbnail Figure 3

Structure of the NN, images are consisting of 8bit images with values from 0 to 1. The resolution is 32 × 32 and 150 × 150 for phase masks and intensity images, respectively.

With the single-layer perceptron layout, training converges within several minutes on consumer hardware for both NNs.

After training the NN, a digital twin of the MPLC setup is created called Model-NN. Consequently, the Model-NN includes all misalignments and tolerances of the setup. In a second step, the Model-NN is kept static and another single-layer NN called Actor-NN is trained controlling the model. The ultimate predictions provide phase masks that are required for the desired light shaping task. This intelligent approach allows us to consider experimental tolerances already in the calibration procedure, which advances the MPLC setup.

2.1 Model-NN mimicking the MPLC setup

The first step of the Actor-Model approach comprises training the Model-NN to create the digital twin. For this task, representative training data is necessary. For our experiments, we consider the setup shown in Figure 1. Two incoherent Gaussian input spots of 640 nm wavelength enter the MPLC. Mirror and SLM are separated with d M ≈ 3 cm distance. After two reflection passages, the output is imaged to a camera (d ≈ 1.5 cm) by a 4f-telescope capturing an intensity image. On the SLM, two-dimensional 256 × 256 pixel, 8 bit phase masks are displayed. In our investigations, SLM pixels allocated to the second reflection passage, i.e. the passage closer to the camera, are active. The SLM pixels allocated to the first passage are set passive. To reduce training effort, groups of 8 × 8 bit neighboring SLM pixels are assigned to one macropixel. Thus, the training data for the Model-NN consists of 32 × 32 pixel phase masks and the corresponding 150 × 150 pixel intensity images, as shown in Figure 3 and Figure 2a. The phase values have to be set within the interval ( 0 ; π 2 ] $ \left(0;\frac{\pi }{2}\right]$ to avoid neighboring fields having conjugate phase [28]. In Figure 4, the training procedure on experimental data is shown. As a result, the Model-NN can predict the MPLC behavior with Γ = 0.94 calculated on average between prediction and ground truth for training data and Γ = 0.82 for test data, respectively. The investigations shown are performed on a single input beam in a first step. This way it can be shown, that NNs can learn the MPLC behaviour in general.

thumbnail Figure 4

Training progress of both Model-NN and Actor-NN. Experimental data (i.e. SLM phase masks and intensity camera images) is used for the Model-NN, while EMNIST data base is used for the Actor-NN. We used 2k data each, where 1600 samples are used for training and 400 for test, respectively. Both NNs comprise of a single-layer structure with sigmoid activation function. MSE is used as loss function with adam optimizer, wheres fidelity is used as performance indicator. Convergence is observed in all scenarios that are training (solid) and test data (dashed) for Model-NN (red) and Actor-NN (blue), respectively.

When considering a second beam at the input, another independent network part has to be created for instance by training another Model-NN. For multi-beam configurations, it is crucial that all output beams have a common overlap region which defines the area where output modes are generated. This constraint can introduce special requirements to the SLM phase masks, especially for many input beams, which will be discussed in Section 3. In Figure 5, intensity heatmaps of a dual-input beam configuration are shown. The heatmaps are calculated from 1k synthetic data, when simulating the MPLC with free-beam propagation using angular spectrum [30]. The images in Figure 5a and Figure 5b result from summing up the intensity images after displaying 1k different random SLM phase masks. In Figure 5c, the overlap between both spots, i.e. heatmaps is shown. This defines the region where output modes can be generated in a later dual-beam MPLC.

thumbnail Figure 5

Example of dual-input beam configuration. Simulation data is shown. (a) and (b) show heatmaps of 1k images using 1k different random SLM phase masks. (c) overlap between (a) and (b).

2.2 Actor-NN controlling the model

In the previous step, the Model-NN is trained mimicking the experimental setup. Now we will use another NN, i.e. Actor-NN to control the model. In this step, the trained Model-NN is frozen, as shown in Figure 2b. As training the Model was performed on all-experimental data, it contains knowledge on the system. This is used in the offline training of the Actor-NN to predict the required phase masks to perform the desired beam shaping task. The Actor-NN architecture is again the single-layer perceptron shown in Figure 3. The input of the Actor-NN can be any desired intensity distribution. We use 150 × 150 pixel, 8 bit intensity samples of the EMNIST data base [29]. In Figure 6 (left column), three samples are shown. The corresponding phase mask prediction of the Actor-NN (see Fig. 6 middle left column) is forwarded to the frozen Model-NN which predicts the MPLC output beam intensity. In the middle right column of Figure 6, three predictions from the Model-NN are shown. MSE again defines the loss function optimizing the Actor-NN to control the Model-NN which represents the experimental setup.

thumbnail Figure 6

Results from the Actor-Model approach. All samples shown are images from the EMNIST data set. Left column: Ground truth images used as Actor-NN input. Middle left: phase mask predictions resulting from the Actor-NN. Middle right: predictions from the Model-NN after the Actor-NN was trained. The correlation between Model-NN prediction and ground truth is Γ ≈ 0.7. Right: experimental results. Using the trained Actor-NN, SLM phase masks are generated that are employed for driving the MPLC. The images shown are camera recordings capturing the MPLC output. Note, that a single-beam and single-passage configuration is considered. The correlation between experiment and Model-NN prediction is Γ ≈ 0.65.

After training of the entire Actor-Model structure (see Fig. 2b), the Actor-NN is frozen and is used to generate phase masks for the experiment, as shown in Figure 2c. The phase masks provided by the Actor-NN are displayed on the SLM to run the MPLC. In Figure 6 (right column), three camera images capturing the MPLC output are shown. The images are taken after the setup was calibrated with the approach introduced. We achieve a correlation between the ground truth and the Model-NNs prediction of Γ ≈ 0.7. For the measured intensity we achieve a correlation of Γ ≈ 0.65 compared to the Model-NNs prediction and Γ ≈ 0.6 to the ground truth.

3 Discussion

Our results show that the smart Actor-Model approach enables targeted shaping of light beams that are input to an MPLC device. With this calibration method, we treat the optical setup as a black box. Thus, we do not necessitate pixel-precise alignment in the experiment to match offline calculation, such as the WMA. In turn, we train a digital twin of the MPLC setup called Model-NN that considers knowledge of the experimental environment reducing misalignment. Therefore, proper training data of the setup needs to be created. Training data of the Model-NN comprises SLM phase masks at the input and corresponding intensity camera images at the output. For the actor, training is done by using EMNIST data set as the ground truth. The prediction of the Actor-NN during training is directly connected to the input phase mask of the fixed Model-NN, predicting the systems output intensity. The Actor-NN is trained by comparing the predicted intensity with the ground truth images. The Actor-NNs performance is limited by the performance of the Model-NN. Additionally the Actor can only provide phase masks, to shape a certain area on the camera, which is mainly the center. By reducing the ground truth intensity to this area, the performance is increased significantly to Γ ≈ 0.7, where Γ ≈ 0.82 is the maximal achievable correlation from the Model-NNs performance for unknown data.

In our investigations, we use simple single-layer perceptrons for a dual-passage configuration. Due to the exponential parameter increase in fully-connected layers when increasing the number of input / output pixels, more sophisticated NN architectures are required, e.g. DenseNet or MTNet [31, 32]. With our approach, offline training of both, the Model- as well as the Actor-NN is possible. This reduces the on-setup time for training data acquisition. In principle, other online training techniques like reinforcement learning [33, 34] can be employed to control the system. However, such approaches require increased on-setup time suffering from instabilities such as mechanical drifts or fluctuations. This is particularly critical if interferometric training data, e.g. digital holography [35], must be acquired for complex mode multiplex. We use MSE as loss function during the training of the NNs. However, a custom loss function of multiple weighted measures, e.g. SSIM or Kullback-Leibler divergence, may improve our results significantly, as this allows to trade-off between spatially and probability distributed features.

For performing mode generation with the system shown, the mode profiles must be used as ground truth. This requires complex training data. So far, however, intensity-only images have been used for both model and actor. For mode generation, the system must therefore be extended by an interferometer. The NN architecture should be complemented by complex-valued NNs [36], or by another NN path carrying phase information [28]. After calibration, modes can be launched by directing the output beam to an actual FMF. The FMF input facet is placed to the position defined by the camera sensor in Figure 1, or to an image of it.

The results shown in this work are produced in single-beam and single-passage configuration. However, to shape coaxial output modes in a multi-beam configuration, all output beams must overlap in a certain region, as shown in Figure 5 for two beams. For all-random phase masks, the overlap area is rather small. In order to increase overlap, or to allow overlapping of multiple beams, targeted shifting can be applied by for instance displaying phase tilts on the SLM. Such patterns are included in Fourier domain phase masks [37, 38], where a pixel-shift induces a tilt in the phase pattern.

In multi-beam configurations, multiple independent network structures, such as multi-Model-NNs have to be trained that mimick the propagation properties of independent optical tributaries. In such arrangements, the Model-NN approach shown in this work can straightforwardly be adopted. The NN architecture in a multi-beam arrangement should comprise independent sub-NNs for each beam in both Actor-NN and Model-NN. Since in each passage all beams share the same phase mask, the Actor-NN outputs must be combined, training on a common phase mask. In the solution shown here, we predict a phase mask for one passage that is used as Actor-NN output, i.e. Model-NN input. However, for multi-passage configurations, all SLM pixels referring to different passages can be treated as one accumulated phase mask.

Adding tributaries and thus increasing the number of input beams requires increasing the number of reflection passages. According to a fair estimate with WMA, N passages are required for N input spots [18]. Upscaling the MPLC necessitates also higher resolution of the Actor-Model-NN architectures, including SLM phase masks and camera images serving as input / output of the Model-NN. Here, we used 32 × 32 pixel phase masks and 150 × 150 pixel camera images. In our investigations, we used 8 × 8 macropixels on the SLM. In total, a 256 × 256 pixel SLM window is used. These dimensions result from a consideration of how much a phase change on the SLM induces a meaningful change on the camera image when generating training data for the Model-NN. If higher resolution SLM windows, i.e. smaller macropixels, are used the resolution on the camera should be scaled accordingly to capture valuable information.

4 Conclusions

We have demonstrated an intelligent approach to calibrate an MPLC device using experimental data. Although we treat the entire light shaping system as black box, delicate knowledge about the experimental behavior is gained by using machine learning algorithms with Γ ≈ 0.82. Here, we have shown that the Actor-Model approach is feasible for online calibration of an all-optical mode multiplexer based on MPLC. In contrast to an offline calculation of SLM phase masks, our approach does not suffer from mismatches between algorithm and experiment reducing the alignment effort dramatically. This is particularly beneficial for the employment of low-complex SDM networks or the transmission of fragile quantum states.

Conflict of interest

The authors declare no conflict of interest.

Acknowledgments

We thank the German Research Foundation (DFG) for support with the project (CZ 55/42-2), the Federal Ministry of Education and Research of Germany in the programme of “Souverän. Digital. Vernetzt.” Joint project 6G-life, project identification number: 16KISK001K and joint project QUIET, project identification number: 16KISQ092, the UK Research and Innovation (UKRI) for support through Future Leaders Fellowship MR/T041218/1 and Nektarios Koukourakis and Polina Bayvel for valuable discussions. Preliminary results were presented at the general congress ICO-25-OWLS-16 [39]. The source code will be deployed upon reasonable request.

References

  1. Baig M.M., GholamHosseini H., Connolly M.J. (2015) Mobile healthcare applications: system design review, critical issues and challenges, Australas Phys. Eng. Sci. Med. 38, 1, 23–38. https://doi.org/10.1007/s13246-014-0315-4. [CrossRef] [Google Scholar]
  2. Saranya P., Asha P. (2019) Survey on big data analytics in health care, in 2019 International Conference on Smart Systems and Inventive Technology (ICSSIT), pp. 46–51. [CrossRef] [Google Scholar]
  3. Uusitalo M.A., Rugeland P., Boldi M., Calvanese Strinati E., Carrozzo G., Demestichas P., Ericson M., Fettweis G., Hamon M.-H., Latva-aho M., Martrat J., Pärssinen A., Richerzhagen B., Sabella D., Schotten H.D., Serrano P., Stea G., Svensson T., Ustundag Soykan E., Wikström G., Ziegler V., Zou Y. (2021) Hexa-X, The European 6G flagship project”, EuCNC 2021, in 2021 Joint European Conference on Networks and Communications & 6G Summit (Eu-CNC/6G Summit), pp. 580–585. [CrossRef] [Google Scholar]
  4. Ellis A.D., Zhao J., Cotter D. (2009) Approaching the non-linear Shan non limit, J. Lightwave Technol. 28, 423–433. [Google Scholar]
  5. Ryf R., Randel S., Gnauck A.H., Bolle C., Sierra A., Mumtaz S., Esmaeelpour M., Burrows E.C., Essiambre R.-J., Winzer P.J., Peckham D.W., McCurdy A.H., Lingle R. (2012) Mode-division multiplexing over 96 km of few-mode fiber using coherent 6 × 6 MIMO processing, J. Lightwave Technol. 30, 521–531. [NASA ADS] [CrossRef] [Google Scholar]
  6. Winzer P.J., Neilson D.T., Chraplyvy A.R. (2018) Fiber-optic transmission and networking: the previous 20 and the next 20 years, Opt. Express 26, 24190–24239. [NASA ADS] [CrossRef] [Google Scholar]
  7. Richardson D., Fini J., Nelson L (2013) Space Division Multiplexing in optical Fibres, Nature Photon. 7, 354–362. [NASA ADS] [CrossRef] [Google Scholar]
  8. Ferreira F.M., Barbosa F.A., Ruocco F.A., Lo M.C. (2022) Scaling up SDM transmission capacity, in 2022 IEEE Photonics Conference (IPC), pp. 1–2. [Google Scholar]
  9. Ferreira F.M., Fonseca D., da Silva H.J.A. (2013) Design of few-mode fibers with M-modes and low differential mode delay, J Lightwave Technol. 32, 353–360. [Google Scholar]
  10. Rademacher G., Puttnam B.J., Luís R.S. (2021) Peta-bit-per-second optical communications system using a standard cladding diameter 15-mode fiber, Nature Commun. 12, 4238. [NASA ADS] [CrossRef] [Google Scholar]
  11. Rademacher G., Puttnam B.J., Luís R.S., Eriksson T.A., Fontaine N.K., Mazur M., Chen H., Ryf R., Neilson D.T., Sillard P., Achten F., Awaji Y., Furukawa H. (2012) Degenerate mode-group division multiplexing, J. Lightwave Technol. 30, 3946–3952. [NASA ADS] [CrossRef] [Google Scholar]
  12. Leon-Saval S.G., Fontaine N.K., Salazar-Gil J.R., Ercan B., Ryf R., Bland-Hawthorn J. (2014) Mode-selective photonic lanterns for space-division multiplexing, Opt. Express 22, 1036–1044. [NASA ADS] [CrossRef] [Google Scholar]
  13. Labroille G., Denolle B., Jian P., Genevaux P., Treps N., Morizur J.-F. (2014) Efficient and mode selective spatial mode multiplexer based on multi-plane light conversion, Opt. Express 22, 15599–15607. [NASA ADS] [CrossRef] [Google Scholar]
  14. Mounaix M., Fontaine N.K., Neilson D.T., Ryf R., Chen H., Alvarado-Zacarias J.C., Carpenter J. (2020) Time reversed optical waves by arbitrary vector spatiotemporal field generation, Nature Commun. 11, 1–7. [NASA ADS] [CrossRef] [Google Scholar]
  15. Czarske J.W., Haufe D., Koukourakis N., Büttner L. (2016) Transmission of independent signals through a multimode fiber using digital optical phase conjugation, Opt. Express 24, 15128–15136. [NASA ADS] [CrossRef] [Google Scholar]
  16. Rothe S., Koukourakis N., Radner H., Lonnstrom A., Jorswieck E., Czarske J.W. (2020) Physical layer security in multimode fiber optical networks, Sci. Rep. 10, 2740. https://doi.org/10.1038/s41598-020-59625-9. [NASA ADS] [CrossRef] [Google Scholar]
  17. Rothe S., Besser K.L., Krause D., Kuschmierz R., Koukourakis N., Jorswieck E., Czarske J.W. (2023) Securing data in multimode fibers by exploiting mode-dependent light propagation effects, Research 6, 0065. https://doi.org/10.34133/research.0065. [NASA ADS] [CrossRef] [Google Scholar]
  18. Fontaine N., Ryf R., Chen H., Neilson D., Carpenter J. (2017) Design of high order mode-multiplexers using multiplane light conversion, in 2017 European Conference on Optical Communication (ECOC), pp. 1–3. [Google Scholar]
  19. Fontaine N.K., Ryf R., Chen H., Neilson D.T., Kim K., Carpenter J. (2019) Laguerre-Gaussian mode sorter, Nature Commun. 10, 1–7. [NASA ADS] [CrossRef] [Google Scholar]
  20. Hashimoto T., Saida T., Ogawa I., Kohtoku M., Shibata T., Takahashi H. (2005) Optical circuit design based on a wavefront-matching method, Opt. Lett. 30, 2620–2622. [NASA ADS] [CrossRef] [Google Scholar]
  21. Fang J., Bu J., Li J., Lin C., Kong A., Yin X., Luo H., Song X., Xie Z., Lei T., Yuan X. (2021) Performance optimization of multiplane light conversion (MPLC) mode multiplexer by error tolerance analysis, Optics Exp. 29, 37852–37861. [NASA ADS] [CrossRef] [Google Scholar]
  22. Borhani N., Kakkava E., Moser C., Psaltis D. (2018) Learning to see through multimode fibers, Optica 5, 960–966. [NASA ADS] [CrossRef] [Google Scholar]
  23. Zhu C., Chan E.A., Wang Y., Peng W., Guo R., Zhang B., Soci C., Chong Y. (2021) Image reconstruction through a multimode fiber with a simple neural network architecture, Scientific Rep. 11, 1–10. [NASA ADS] [CrossRef] [Google Scholar]
  24. Pohle D., Rothe S., Koukourakis N., Czarske J. (2022) Surveillance of few-mode fiber-communication channels with a single hidden layer neural network, Opt. Lett. 47, 5, 1275–1278. [NASA ADS] [CrossRef] [Google Scholar]
  25. Sun J., Wu J., Koukourakis N., Cao L., Kuschmierz R., Czarske J. (2022) Real-time complex light field generation through a multi-core fiber with deep learning, Scientific Rep. 12, 1–10. [NASA ADS] [CrossRef] [Google Scholar]
  26. Teğin U., Rahmani B., Kakkava E., Borhani N., Moser C., Psaltis D. (2020) Controlling spatiotemporal nonlinearities in multimode fibers with deep neural networks, APL Photonics 5, 030804. https://doi.org/10.1063/1.5138131. [CrossRef] [Google Scholar]
  27. Lin X., Rivenson Y., Yardimci N.T., Veli M., Luo Y., Jarrahi M., Ozcan A. (2018) All-optical machine learning using diffractive deep neural networks, Science 361, 1004–1008. [NASA ADS] [CrossRef] [Google Scholar]
  28. Rahmani B., Loterie D., Kakkava E., Borhani N., Teğin U., Psaltis D., Moser C. (2020) Actor neural networks for the robust control of partially measured nonlinear systems showcased for image propagation through diffuse media, Nat. Mach. Intell. 2, 403–410. [CrossRef] [Google Scholar]
  29. Cohen G., Afshar S., Tapson J., van Schaik A. (2017) EMNIST: Extending MNIST to handwritten letters, in 2017 International Joint Conference On Neural Networks (IJCNN), pp. 2921–2926. [CrossRef] [Google Scholar]
  30. Matsushima K., Shimobaba T. (2009) Band-limited angular spectrum method for numerical simulation of free-space propagation in far and near fields, Optics Exp. 17, 19662–19673. [NASA ADS] [CrossRef] [Google Scholar]
  31. Rothe S., Zhang Q., Koukourakis N., Czarske J. (2021) Intensity-only mode decomposition on multimode fibers using a densely connected convolutional network, J. Lightwave Technol. 39, 1672–1679. [NASA ADS] [CrossRef] [Google Scholar]
  32. Zhang Q., Rothe S., Koukourakis N., Czarske J. (2022) Learning the matrix of few-mode fibers for high-fidelity spatial mode transmission, APL Photon. 7, 066104. https://doi.org/10.1063/5.0088605. [CrossRef] [Google Scholar]
  33. Luo Y., Yan S., Li H.A., Lai P., Zheng Y. (2020) Focusing light through scattering media by reinforced hybrid algorithms, APL Photon. 5, 016109, https://doi.org/10.1063/1.5131181. [CrossRef] [Google Scholar]
  34. Schmidt K., Koukourakis N., Czarske J.W. (2022) Assignment of focus position with convolutional neural networks in adaptive lens based axial scanning for confocal microscopy, Appl. Sci. 12, 661. https://doi.org/10.3390/app12020661. [Google Scholar]
  35. Cuche E., Bevilacqua F., Depeursinge C. (1999) Digital holography for quantitative phase-contrast imaging, Optics Lett. 24, 291–293. [NASA ADS] [CrossRef] [Google Scholar]
  36. Barrachina J.A. (2022) NEGU93/cvnn: Complex-Valued Neural Networks version v2.0. Nov. 2022. [Google Scholar]
  37. Loterie D., Farahi S., Papadopoulos I., Goy A., Psaltis D., Moser C. (2015) Digital confocal microscopy through a multimode fiber, Optics Express 23, 23845–23858. [NASA ADS] [CrossRef] [Google Scholar]
  38. Koukourakis N., Wagner F., Rothe S., Karl M.O., Czarske J.W. (2022) Investigation of human organoid retina with digital holographic transmission matrix measurements, Light Adv. Manuf. 3, 211–225. [Google Scholar]
  39. Pohle D., Barbosa F.A., Ferreira F.M., Czarske J., Rothe S. (September 2022) Intelligent self calibration tool for adaptive mode multiplexers using multiplane light conversion, in General Congress ICO-25-OWLS-16, Dresden, Germany, September 2022. https://ico25.org/. [Google Scholar]

All Figures

thumbnail Figure 1

Optical MPLC setup considered in our work. Two spatially seperated Gaussian input spots are incident to the SLM. After two reflection passages between SLM and mirror (M), the output beams are imaged by a 4f-telescope onto a camera where intensity images are recorded. Both the FMFs facet and the camera are at a distance d ≈ 1.5 cm from the SLM. A smart calibration based on artificial intelligence (AI) is implemented to generate proper SLM phase masks in order to shape desired output modes.

In the text
thumbnail Figure 2

Structure of the envisioned NN architecture. (a) Training of a single-layer Model-NN. Training data consists of 2k uniformly distributed random SLM phase masks and the corresponding intensity images measured at the MPLC output. This generates a digital twin of the MPLC setup. (b) Another single-layer called Actor-NN is trained on 2k intensity images according to the EMNIST data set. The Actor-NN is used to predict phase masks, which are the input for the Model-NN. When training the Actor-NN, the Model-NN is fixed. (c) The predicted phase mask of the trained Actor-NN is applied to the SLM and the intensity is measured. We achieve a correlation of Γ = 0.65 compared to the Model-NNs predicted intensity.

In the text
thumbnail Figure 3

Structure of the NN, images are consisting of 8bit images with values from 0 to 1. The resolution is 32 × 32 and 150 × 150 for phase masks and intensity images, respectively.

In the text
thumbnail Figure 4

Training progress of both Model-NN and Actor-NN. Experimental data (i.e. SLM phase masks and intensity camera images) is used for the Model-NN, while EMNIST data base is used for the Actor-NN. We used 2k data each, where 1600 samples are used for training and 400 for test, respectively. Both NNs comprise of a single-layer structure with sigmoid activation function. MSE is used as loss function with adam optimizer, wheres fidelity is used as performance indicator. Convergence is observed in all scenarios that are training (solid) and test data (dashed) for Model-NN (red) and Actor-NN (blue), respectively.

In the text
thumbnail Figure 5

Example of dual-input beam configuration. Simulation data is shown. (a) and (b) show heatmaps of 1k images using 1k different random SLM phase masks. (c) overlap between (a) and (b).

In the text
thumbnail Figure 6

Results from the Actor-Model approach. All samples shown are images from the EMNIST data set. Left column: Ground truth images used as Actor-NN input. Middle left: phase mask predictions resulting from the Actor-NN. Middle right: predictions from the Model-NN after the Actor-NN was trained. The correlation between Model-NN prediction and ground truth is Γ ≈ 0.7. Right: experimental results. Using the trained Actor-NN, SLM phase masks are generated that are employed for driving the MPLC. The images shown are camera recordings capturing the MPLC output. Note, that a single-beam and single-passage configuration is considered. The correlation between experiment and Model-NN prediction is Γ ≈ 0.65.

In the text

Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.

Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.

Initial download of the metrics may take a while.