Saturday, September 22, 2007

Imperfect Information and Suboptimal Design

I have been been doing online research on digital cameras in an attempt, so far unsuccessful, to persuade myself to replace my beloved pocket digital camera with the latest model in the same line. In doing so I came across an interesting analysis of what is wrong with the current generation of small cameras and why.

Small cameras have small sensors, which leads to a tradeoff between pixel count and sensitivity. The designer can divide the sensor into lots of individual units, giving the camera a high pixel count but making each unit small. Or he can design it with fewer but larger units. Smaller units are less sensitive, meaning that they either don't pick up an image with low light or pick it up with noise, giving a poor quality image.

The number of pixels in the image a camera produces is close to, although not entirely, an objective fact, routinely included in brief summaries of a camera's characteristics. The sensitivity is a subtler issue. A manufacturer or reviewer can report the highest sensitivity—the equivalent of ASA numbers on film—that the camera can operate at. But the manufacturer can increase that number without improving the camera by accepting more noise in the images. A good review will include a discussion of the quality of the image, and examples, but most consumers are unlikely to look for or use that sort of detailed information.

That gives the manufacturer an incentive to design the camera with more than the optimal pixel count at the cost of lowered sensitivity and lowered quality of image. He is trading off two desirable features only one of which is easily observed by the consumer, hence will put too much weight on that one. Arguably the result is a seven or eight megapixel camera—higher resolution than most photographers most of the time have any use for—that takes lower quality pictures than an otherwise similar five mexapixel camera

I don't know enough about cameras or the camera industry to judge whether the analysis I have just offered is true but I see no reason why it couldn't be. That raises an interesting question. Are there other products for which the same problem exists? Are there other situations where a producer must trade off two desirable characteristics, one of which is much more readily measured by consumers than the other, and produces as a result a suboptimal design?

One possibility that occurs to me is the tradeoff in automobile design between miles per gallon, achieved in part by making a car lighter, and ability to survive a crash. Other suggestions?


Anonymous said...

A very general one is: price and quality.

Brandon Berg said...

Higher clock speed versus fewer cycles per instruction in computer CPUs.

Anonymous said...

Computers: Higher CPU clock speeds vs. faster bus speeds. The consumer can see all the numbers, but isn't informed enough to know that they'd get better performance from a faster bus, so we end up with CPUs that spend much of their time waiting on the bus.

Sports cars: More Power vs. Less weight. Weights are rarely mentioned, but horse power always is, despite the fact that a better measure of performance is the power/weight ratio.

One theory is that the high investment or high margin items are the ones that get emphasized due to business interests and company politics.

Chris Bogart said...

Software: feature count versus usability. Product reviewers love little feature tables, but glomming lots of features on, which may work poorly together, can hurt usability.

HOA fees: some homeowner's associations will have low monthly fees, supplemented by lots of special assessments, rather than budgeting properly for maintenance that needs to be done. The monthly HOA fee is included in real estate listings, the special assessments are not. So it makes the HOA look less onerous to buyers.

Arnaud M. said...

Computer monitors: Size (in inches) vs. resolution. Cheaper screens always have lower-than-standard resolution.

Sciencebzzt said...

an aside:
I've found that this site is the most comprehensive camera site on the internet. this isn't a spam, I just thought you might find it helpful. The reviews are in-depth, cover everything you can think of, with examples of pictures taken from each camera and full specs. Plus they have articles about what makes one digital camera take better pictures than some other.

Have you decided on a suitable sub-title to "Future Imperfect" (or whatever it will be called) yet?

Jonathan said...

Surely it's a very general problem that the marketing and even the reviews of any kind of product tend to focus on certain familiar characteristics at the expense of less familiar but perhaps also important characteristics. I suppose a company whose product does well in the less familiar areas has an incentive to try to make them more familiar; but re-educating the public is probably more difficult and expensive than redesigning the product to look good by conventional criteria.

Perhaps the only hope is that diligent reviewers will identify product characteristics that deserve more attention.

Jonathan said...

By the way, thanks for the interesting insight into camera design. I wasn't aware of the trade-off you describe.

I now use a Nikon D50 (SLR). It's heftier than a pocket camera, but I find it works so much better than any of the small cameras I tried before.

Mike Huben said...

Packaging versus package disposal.

Anonymous said...

Printers comes to mind.

When my wife had a newsletter, she completed a single reproducable newsletter, which was reproduced in quantity by the printing company. We noticed terrible results with photographs when reproduced. The original was clear, yet the copies had poor quality.

After research, we found that many printers print very good pictures, but when you examined them with a magnifying glass, the small dots weren't clear at the edges. (I think the term is dithering) The only good printer for reproducable photos had small clearly defined dots when examined. It really didn't matter how many there were, if they were clear. Otherwise the DPI was irrelevent in this application.

The sacrifice in quality dots allowed lower prices. To the naked eye, the pages were clear and "good enough". To the professional, the quality was sacrificed.

Will McLean said...

Warship design. In the late 19th and the 20th c. gun caliber, armor thickness and trial speed were the obvious measures of quality. Sustained speed in a seaway, seaworthiness, damage control and habitability were less obvious.

Anonymous said...

I suspect that almost any case of "externalities" or "hidden costs" in any Econ book can be viewed as an example of this phenomenon: if you have a choice between designing things to reduce a hidden cost to the consumer, and designing things to reduce an obvious cost to the consumer, the latter will get you more consumers.

For a classic example, assume for the sake of argument that pollution from a widget factory causes significant health costs, distributed across the nearby population. Any sensible factory owner will put more effort into reducing the retail price of the widgets than into reducing the pollution produced by the factory, even if the health savings to neighbors would be greater than the purchase-price savings to customers.

Arthur B. said...
This comment has been removed by the author.
Anonymous said...

Sounds akin to the process that gave us the peacock's tail, somehow.

Roger Collins said...

I think the influence of early adopters, professional users, opinion leaders, experts, publications, etc. provide enough feedback into the system to cause reputation-sensitive manufacturers to still be motivated to make the best overall product they can make. Note Mhz of CPUs went down in laptops from one generation to the next despite its being critical and trivial measure of performance to consumers.

Brands also address this. A brand's reputation would suffer if it attempted to use "spin" alone to push products. Buy a good brand and you should not get severely ripped off.

How many hours of research is profitable for selecting a camera for individual use?

Beastin said...

Maintenance versus sticker cost is probably pretty general as well.

Incidentally, a few months ago there was a New York Times OpEd, "Breaking the Myth of Megapixels", that concluded that anything over 5 megapixels was overkill for casual photographers.

Anonymous said...

For cameras there is another critical dimension: the lens. There is no simple figure of merit for a lens, and good lenses are generally larger (due to physics) and expensive. So the lower end camera vendors just ignore the lens.

A few may note plastic vs glass lenses, number of lens components, etc., but it is generally when you get up to the expensive cameras that you find the extensive details regarding lens construction and characteristics.

Eric H said...

Actually, if the manufacturer follows the ISO standard for reporting the speed, it is based on the S/N ratio. You are supposed to report it out specifying whether you use S/N=10 (first usable image), S/N=40 (first excellent image), or for saturation, and also by the light source used. Unfortunately, and this has been a complaint among photographers, they don't use the standard, so people have been testing them and publishing results independently. I believe you can find results at Norman Koren's website or one of the related forums.

For many consumers, what matters most is whether they can get an acceptable picture at 4 x 5, 8 x 10, or filling their monitor. In those cases, it is better to have more pixels to mask the "jaggies" along well-defined edges. So long as there is enough light (flash, outdoor daylight), the sensitivity and noise issues can be addressed, so I'm not sure there is as much tradeoff as you claim.

A more relevant tradeoff might be pixel count vs. fixed pattern and cross-talk noise or pixel size vs. fill factor. Resolution is roughly related to the size and number of pixels, but not exactly. You can calculate a theoretical limit based on those measures, but noise will degrade below that limit. You will rarely if ever see specifications on those noise sources. Fill factor is the ratio of the sensitive area of the pixel to the total area of each pixel (accounting for the circuitry that moves the signal off the active area). CMOS sensors have much worse fill factor than CCD sensors (roughly 40% vs. 90-99% in CCD sensors). FF is rarely stated in consumer literature, so it is difficult to compare apples to apples. The consumer is stuck with pixel counts and possibly pixel sizes, but not the other information.

One problem with all of this is that you could easily inundate the consumer with the esoterica. For that reason, product testing is preferred to reading manufacturer's literature. One picture is literally worth 30 or so pages of brochure.

Andrew said...

Peter H writes:
CMOS sensors have much worse fill factor than CCD sensors (roughly 40% vs. 90-99% in CCD sensors)

Of course, then one would say that the best high end dSLRs, used by the large majority of pro sports photographers, are Canon dSLRs, which all use CMOS sensors! The more a consumer looks into the details, the more a consumer will get confused.

Sample photos are great, but under the right conditions, a good photographer can make a picture with ANY kind of camera. So, manufacturer-provided samples are not good to use. Camera reviews which provide sample photos of the same places under similar conditions are very useful, though.

Anonymous said...

Nothing beats going to a decent camera shop that will let you take your own sample pictures, then view them.

That way you can directly compare images from different cameras and what the photo is of.

As an aside: dSLRs have a _much_ larger sensor area than compacts.