THE LYNX A Camera does object modeling as well as scene modeling and motion capture.
A person goes to college. Sometimes, graduate school. Then, armed with the knowledge and experience obtained from the classroom, the student seeks employment in what is today a very tight job market. At least, that is the traditional path most take from student to professional. However, for two particular students from the University of Texas at Austin, there was a different option worth exploring: Starting their own company.
The firm is called Lynx Laboratories, founded several months ago by Chris Slaughter (CEO) and Jeff Mahler (senior engineer), along with Dustin Hopper (senior software engineer) and Sriram Vishwanath (advisor). Also joining the team is Nick Shelton (computer vision), Albert Rondan (graphics), and Kyle Cox and Larry Walker (advisors). The company is full of young, eager students/recent grads, all but one of whom hails from the University of Texas or are experienced advisors from the university.
The startup was hatched from a computer vision research group at the University of Texas. It was there where engineering grad student Slaughter had the opportunity to work in the school’s research laboratory, starting up the UT Perception Lab. The lab, which involved a handful of people, was trying to solve the problem of inferring image-based data – such as shape and motion – from a camera feed.
“It started out as a computer research problem, and we had some breakthroughs there, and after demonstrating what the technology was capable of, people in the community started reaching out to us, saying that this would be really great for industrial inspection, architectural surveys, and visual effects, things like that,” says Slaughter. “After hearing the really high level of interest from the community, we decided [pushing our vision] would be better accomplished as a commercial company than as a research lab.”
At the university, word of Slaughter’s lab project had spread throughout the various disciplines, from the architectural department to the film department, and then via friends and then friends of friends. The project developed organically from there.
Today, Lynx Laboratories has transformed that project/lab into a product: the Lynx A Camera, a 3D capture device that can digitize the shape and motion of an object the camera is trained on. The group calls it “structural capture and next-generation photography.”
According to Slaughter, the core research was done at the university, but the group has taken that further with an invention spawned from the underlying technology. “Our company is focused more on productization now,” says Slaughter, who counts his former PhD supervisor, Vishwanath, as the company’s secret weapon. “He is an extremely talented mathematician, and his contribution has been in solving theoretical-based problems that have popped up,” Slaughter says.
Lynx Laboratories opened its offices in Austin last summer, and with a National Science Foundation grant early this year, is now self-sufficient. To help secure funding for the camera, the team turned to one of the more popular methods of funding at the present time: Kickstarter. In March, Lynx Laboratories had met its goal of $50,000. In fact, it had exceeded that with $86,733 pledged by 219 backers – some whom kicked in approximately $1,800 to $2,000 and will receive a camera, and others who gave $1 because they were excited to see what the group was trying to accomplish. (See Lynx Laboratories’ Kickstarter page at www.kickstarter.com/projects/915328713/lynx-a-camera.)
LYNX LABORATORIES has made scene modeling fast and inexpensive.
When asked when he and his group decided to pursue a commercial venture, Slaughter spoke with the passion of a new CEO. “If you take entrepreneurial classes, the first thing they tell you is to learn the market, to find a problem before you write a single line of code. That is not what happened here,” he says. “Our background is in raw sciences, so our impetus for the lab was not to become billionaires, but for people to come in and say, ‘I would really enjoy using this.’ ”
Slaughter continues: “The university set us up to demonstrate the technology, to see if it would work. Lynx Laboratories refined it so it is rock solid and software based, so it is stable, runs great, and ready for productization. Almost all the work was done since we left the university. One of the reasons is because you don’t know all that you need to do [for the product] until you talk to customers and they say, ‘You need to focus on this or that.’ For example, the object modeling, scene modeling, and motion-capture [capabilities] were all [devised] at the company.”
Earning an A
What exactly is the Lynx A Camera? It’s not a conventional camera, but it sort of works like one, so it is easy to use. “If you can use a point-and-shoot Nikon, you’ll find the Lynx even easier to use,” the team states on its website. However, instead of outputting 2D images like a typical camera, the Lynx A produces 3D models of whatever the user points the camera at, and “does so right before your eyes,” says Rondan. In summary, the camera is for people who need 3D content.
The Pro model does three things: scene modeling, object modeling, and motion capture. The goal is to capture high-quality content that would otherwise be made by hand today – “much like what the 2D camera did for oil painting,” Lynx Laboratories’ product summary notes. Five-hour 3D modeling tasks can be done in just seconds, animators can have a motion-capture studio in their backpack, reducing the total amount of time required to animate, and directors can use accurate 3D models to plan shots during pre-visualization.
The most disruptive feature of the device is its price. The Basic Lynx A Camera for object modeling only is priced at $1,999, while the Pro model, which performs object and scene modeling, as well as motion capture, costs $2,999.
SCENES are output with an adjustable polygonal count.
As Slaughter notes, if a person cobbled together all the hardware and software needed to accomplish these tasks, it would cost “a couple hundred grand.” For instance, Lidar scanners are priced in the $40,000 range and results require extensive post-processing. In comparison, the Lynx Laboratories device sells for about the same price as a full-framed DSLR, “making it a serious value for small companies and innovators trying to break into these technologies.” And the images are built in minutes, with no post-processing required.
Rondan points out that there are so many potential applications for the device, including 3D printing: The camera can be used to quickly capture a 3D model for 3D printing, without having to learn complex modeling software. For visual effects, it can be used for compositing. For architects, it can be used to survey properties and take measurements. A plastic surgeon can use it to obtain before and after models of a patient. “The options are open in places where before, working with 3D just wasn’t feasible” he adds.
To use the technology, a person first captures models and motions by pointing and shooting with the camera. An additional recorded pass will improve the model results; walking around an area with the camera will extend the model. To “undo” parts of the model, the user simply rewinds. Next, the person previews the results, thanks to the camera’s graphics engine. Last, the results are exported into a program such as Blender’s open-source software or Autodesk’s Maya or Revit, to name a few.
The Lynx A contains 3D sensing, stellar optics, and more. Best of all, it is portable, about the size of a thick tablet (11.5x8x1 inches), and weighs about six pounds. It has specialized, front-mounted optics, similar to an Xbox Kinect. On the front, there is a 14-inch color LCD screen for instant viewing of image results. The models or scenes are navigated using the two joysticks. It also contains an Nvidia graphics card powerful enough for capture and rendering, an Intel Core i5 (2.6Ghz) processor, and high-capacity storage (500gb disk space). There are two USB 3.0 I/O slots with HDMI out, so data can be moved from the Lynx A via a USB stick.
Rondan points out that these are the hardware specifications of the very first prototype. The newest Lynx camera is already 30 percent smaller and 30 percent faster, and specifications continue to change as the team decides on the best hardware for the price.
Slaughter jokingly calls the hardware the “necessary evil” of the technology. Otherwise, a user would need a $5,000 workstation to use the tech, he points out. The magic occurs in the software running inside – the algorithms Lynx uses to translate the captured data.
Thanks to the software, users can capture detailed 3D models of real objects (even people), at one point within a 1x1x1-meter volume, which are accurate to below a centimeter – in color and at full scale (640x480 color resolution) – a process the company likens to “sculpting with a chisel.” As the user walks around the object, the device slowly carves out a watertight 3D model of the object. “The object models’ resolution is about 3mm minimum, so enough to make a detailed shape but not individual strands of hair or eyelids,” says Rondan. (Since the interview, a new sensor and different hardware have enabled a much larger volume and sub-millimeter accuracies.)
The models can then be exported as .obj, .stl., .xyz, or .ply files (or any other file format needed), which can be 3D printed or imported digitally into a video game, for instance; objects from a set can be captured quickly and used for pre-visualization. The Lynx object modeling feature, according to the company, is 100 times faster than modeling by hand. The model forms on the screen during the capture, so the user knows immediately if any detail was missed, and is ready within seconds after pressing Stop.
The camera can acquire either per-vertex color (blended rastering) or UV textures. “People have told us they want less polygons and greater color information. So we are gearing up for what people need,” says Rondan.
Scene modeling enables users to create environments in seconds for previs, 3D walkthroughs, and design tasks. The scene-modeling feature captures detailed 3D meshes of large (and small) environments up to 1,000 square feet without slowing down or affecting accuracy, “painting them on the screen.” The models are optimized to correct for camera motion and to ensure consistent layouts. The imagery is output as a structured polygonal mesh, with an adjustable polygon count depending on the user’s need.
The radial accuracy of the object modeling had been 9.5cm, while the accuracy of scene modeling was .3 to 3cm locally using the Lynx A. More recently, those accuracies have improved by a factor of six, Rondan points out.
A third major function of the Lynx A is for markerless motion capture that can be done anywhere, as simply as using a video camera – and the end result is having an instant motion volume right in front of the camera. Most motions require one or two takes, and the animation is ready to preview right away. The camera works within a 0.7- to 3-meter range, producing animation for 15 joints at a resolution of 2cm, depending on the range of capture. The motion files export in .bvh format.
“You can buy or rent equipment, the big stuff, for $150,000 that will get you 95 percent there; we can get you 90 percent there for far less,” says Slaughter. “Our [technology] runs fast and doesn’t require a lot of expertise to operate. It gives a pretty good start for those who are not James Cameron doing Avatar. If you need perfect results, then there are expensive options out there.”
ABOUT the size of a tablet, the camera provides 3D content for a range of uses.
Kicking Things Up
The team at Lynx Laboratories had taken in all the comments from Kickstarter and other sources to build a product that will deliver what people need. While many have turned to Kickstarter recently to get funding, Slaughter says the Lynx product did not really fit the basic Kickstarter mold in that it is specialized for a specific targeted audience, not a generalized offering for many. “It’s also more expensive by a factor of six from what you typically see on Kickstarter,” he adds. “We knew there was not going to be an explosion [of funding] but thought it was a great opportunity.”
As Slaughter explains, the group could have sought out investors for commercial deals, but didn’t have to do that. “Our biggest goal was to get something out the door sooner rather than later and build a community around the product and hear the great ideas that people have in using it. Kickstarter was a great way to do that. We weren’t using Kickstarter to sell our product, but to engage people in the process, for them to share their ideas. And, it was a great experience for our team.”
With the funding goal reached and exceeded, the group has been working to integrate additional features into the product, many of which were requested in the Kickstarter process. Lynx Laboratories expects to ship the product in June. “Before Kickstarter, we were looking at how to get higher-resolution color onto the models. We have already implemented such a process and also increased the resolution of the 3D data by a factor of six,” notes Rondan. “As long as more than one person is asking for it, we need to build it. I think the backers are already seeing that process happening.”
The biggest technical hurdle, though, was in developing the application, as 3D processing “is just so slow,” says Slaughter. “Using what is basically a 3D modeling program, it just takes so long to render [the imagery]. Imagine building a model and rendering it at 30 frames per second. It is a very ambitious technical problem to solve.” He spent a year in the research lab “getting a handle on that.” Another hurdle was a less technical one: getting a bunch of engineers to build a company. For that, Slaughter credits the assistance of great mentors.
Along the road to launching a company and a product, Lynx Laboratories received encouragement through a number of awards, including Idea2Product Global, Top 10 Dell Innovators, and funding from the National Science Foundation Small Business Innovation Research. For the past several months, the group has been tweaking the camera and showing it at industry events, including SXSW, while preparing the camera for its June rollout. For the time being, Slaughter has pressed the pause button on his PhD so he can focus on the company, while some employees are preparing to graduate in May.
While so many grads across the country will be looking for work this month, this group is well ahead of their peers. In fact, one of them just may be sitting behind a desk interviewing recent graduates for this growing business.
Karen Moltenbrey is the chief editor of CGW.