To start a compact camera lens design with AI, let a trained neural network propose the starting design and let standard optimization finish it. At Opdo, the AI platform for production-ready optics engineering, such a network has been trained, and our AI agents now use it. It gives a lens as sharp as physics allows for 94.5 % of the specifications we tested, in 2 seconds. Sixteen random guesses per specification reach 10.9 %.
A lens design starting point is the first complete recipe for a lens, every curvature, thickness and glass, that an optimizer then refines into the final design.
Why the lens design starting point decides the lens
The starting point decides most of the final quality, and we now get it from a network instead of a patent. Lens optimizers work like a hiker walking downhill in fog. They reach the nearest valley quickly, but they cannot see the deeper valley behind the next ridge.
That is why lens designers start from patents, lens databases or designs they already know. This works, but it needs experience. It is slow when no close design exists, and it ties the result to what is already published.
The lens type is the one in many compact cameras: four glass elements with the aperture, the opening that sets how much light gets in, in front. All eight surfaces are aspheres, shaped beyond a simple sphere as in phone cameras. The network had to cover a whole range of specifications, not one lens.
| Requirement | Range or limit |
|---|---|
| F-number (lower means a brighter lens) | F/2 to F/4 |
| Half field of view (how wide the camera sees) | 10 to 35 deg |
| Lens length relative to focal length | 1.1 to 1.6 x |
| Design wavelengths (three per band) | visible: 486.1, 587.6, 656.3 nm; near infrared: 820, 850, 880 nm |
| Focal length | designed at 5 mm, all lengths scale to the focal length you need; error within 1 % |
| Angle at which light hits the sensor | at most 30 deg |
| Distortion | at most 3 % |
| Color fringing at the image | at most 2 µm, one sensor pixel |
| Brightness at the image corner | at least 40 % of the center |
| Shapes a lens maker can build | minimum thicknesses, at most 0.1 mm off a sphere, no surface steeper than 40 deg |
Fourteen constraints in total, checked for every design.
What we did: a network trained only by a ray tracer
The network learned from a ray tracer alone, with no database of existing lenses. Opdo's AI agents run the optical design and simulation tools engineers already use, take on the repetitive work, and pause for the decisions the engineer makes. The network described here has been trained and tested, and it is now one of the tools our AI agents use on design work.
The network takes a specification and proposes four candidate lenses at once. A differentiable ray tracer, one that can tell the network which way to change each number, traces about 1.2 million rays per training step. It scores each candidate by its spot size, how tightly the lens focuses a point of light, plus a penalty for every broken constraint. That score is the only teacher.
This is the difference to earlier work. Côté, Lalonde and Thibault trained their starting-point network on 150 reference designs from a lens database, and without that data their results got much worse. We only showed the network 5,016 designs that our own optimizer had found from random guesses, to get it going. After that, the ray tracer did the teaching.
Most of the engineering went into keeping training stable. At step 20,000 a sudden jump in how fast the network was allowed to change knocked it away from the designs it had found. We now raise that rate gradually after each restart, and an automatic rollback restores the best version when results blow up.
On the other side, the network was wasting effort. It polished spots to 0.16 µm, a twelfth of a sensor pixel, while learning new specifications slowly. We capped the reward at the diffraction limit, the sharpness physics allows, and let the range grow as fast as the network could follow. This curriculum idea comes from Yang, Fu and Heidrich. The share of sharp designs went from 20 % to 61 % in 4,500 steps.
Chip design took the same route. Google's chip placement network is trained once and then places a new chip in hours, where experts needed weeks to months. Our training took 39 hours on one laptop GPU, done once, which is doable. After that, every new specification costs milliseconds.
Result: 94.5 % diffraction-limited designs in 2 seconds
On 256 fresh specifications the network never saw, its best candidate met all constraints for 99.6 % and was diffraction-limited, as sharp as the physics of light allows, for 94.5 %. The median spot was 0.33 µm, about a sixth of a 2 µm sensor pixel.
| Method | Starts per spec | Meets all constraints | Diffraction-limited | Median spot (µm) | Time for 256 specs |
|---|---|---|---|---|---|
| Network, no polish | 4 | 99.6 % | 94.5 % | 0.33 | 2 s |
| Network + 300 optimizer steps | 4 | 99.6 % | 94.1 % | 0.30 | 111 s |
| Random guesses, 300 steps | 16 | 62.9 % | 10.9 % | 13.8 | 432 s |
| Random guesses, 2,000 steps | 16 | 57.8 % | 10.9 % | 11.8 | 2,879 s (48 min) |
Every method used the same optimizer, ray tracer and constraints. Giving random guesses seven times longer did not raise their success rate at all. Beyond a 25 deg half field they found no sharp design, while the network kept 78 to 100 %. Polishing the network's output barely changed it, so its designs already sit at, or very close to, the bottom of their valley.
Our ray tracer agrees with the open-source tracer Optiland to less than a thousandth of a micrometer, so these numbers are not an artifact of our own code.
Where it breaks: the limit of the lens type
The network misses where this lens type runs out of room, not where the network runs out of skill. A second network covers a far wider range, F/1.2 to F/8, 1 to 40 deg and lengths from 0.5 to 3.0 x the focal length, plus a combined visible and near-infrared band (486.1, 656.3 and 850 nm).
| Wide-range network, by region | Specs | Meets all constraints | Diffraction-limited |
|---|---|---|---|
| All | 512 | 81 % | 66 % |
| Length at least 1 x focal length | 404 | 97 % | 83 % |
| Length 0.5 to 1 x focal length | 108 | 22 % | 4 % |
| F/1.2 to F/2.2 | 73 | 74 % | 22 % |
| Half field 35 to 40 deg | 62 | 68 % | 45 % |
A lens shorter than its own focal length is the clearest case. At the extreme, half of a 5 mm focal length, the lens may be 2.5 mm long. The minimum glass, air, gap to the sensor and filter already take about 2.2 mm.
On the original range, neither polishing nor 16 random guesses with 2,000 steps each rescued any of the 14 lenses the network missed. That points to the lens type, and the fix is a different one, such as an aperture between elements or a fifth element.
What the score does not cover yet: contrast of fine detail (MTF), sensitivity to manufacturing errors and stray light. Glasses are modeled, not picked from a catalog. The exported Zemax files were checked ray by ray in Optiland, not yet in Zemax OpticStudio itself.
The key takeaway is this: the network gives a strong starting point, and the engineer still finishes the design.
Training has not stopped. The network is being trained over a broader range of lens types and specifications, and more features, such as manufacturability, will be trained into it soon.
What it means for you
In a compact camera project, the starting point is usually the slow step and the one that needs the most experience. With the network it takes seconds, so you can repeat it every time the spec changes. You also learn early whether your specification is reachable with this lens type, before anyone orders a prototype.
In a customer project, Opdo's AI agents use this network to explore starting designs. They then optimize the chosen one and check it against your specification in tools such as Zemax, and you decide what goes forward. See how our AI Agents work on design projects, or how we designed a thermal camera lens that stays in focus from -40 to +85 °C.
Frequently asked questions
Can AI design lenses? For this class of lens, yes. A network trained only against a ray tracer proposed diffraction-limited four-element camera lenses for 94.5 % of new specifications, in seconds. The engineer still decides the trade-offs and the tolerances.
How do you find a good lens design starting point without a patent? Train a model against a ray tracer over the specification range you care about, so it learns where good designs sit. It then proposes complete starting designs for a new specification in milliseconds.
Can Opdo use this on my camera project? Yes. Send us your F-number, field of view, length limit and spectral band. Opdo's AI agents propose starting designs and take the chosen one through optimization, with the design handed over as a Zemax file.
Want to design your next compact camera lens with AI? Bring us your use case. We'll deliver a production-ready design. Talk to us.
What would you design with AI agents?
Sources
- G. Côté, J.-F. Lalonde, S. Thibault, "Deep learning-enabled framework for automatic lens design starting point generation," Optics Express 29(3), 2021. Link
- X. Yang, Q. Fu, W. Heidrich, "Curriculum learning for ab initio deep learned refractive optics," Nature Communications 15, 2024. Link
- A. Mirhoseini et al., "A graph placement methodology for fast chip design," Nature 594, 2021. Link
- Optiland, open-source optical design and ray tracing in Python, version 0.6.3. Link