The Uber IQ Camera benchmark is a mobile camera Image Quality (IQ) scoring system that gives an excellent indication of image quality (IQ) based on four “pillars” that everyone can interpret: Daylight, Low-light, Ultrawide, and Zoom photography.

See down below, after the detailed description of our benchmark, the COMPLETE TABLE with the smartphone list, and all their Uber IQ Camera scores (direct link to the table at the bottom of this article).

This Image Quality benchmark comes in addition to our Uber HW Camera, our proprietary algorithm that evaluates the mobile cameras solely based on their hardware’s technical data. To date, we have seen a good correlation between our Uber IQ and our Uber HW camera benchmarks.

Consumers now have an easy and meaningful way to evaluate a phone’s camera performance under real-world conditions. That’s how we review mobile camera quality now.

Consumer Photography research often groups mobile camera users in two categories:

  1. Photo producers (photographer, camera user/buyer)
    1. Sub-group: casual photographers (large consumer majority)
    2. Sub-group: serious hobbyist
  2. Photo consumers who only watch photos, usually without any context of how or why they were taken.

Our mobile camera scoring system and our editorial reviews are written for the casual Photo Producers, even though serious hobbyists should find our conclusions interesting.

000Image Quality score UBER-G SCOREScoring/ranking system name IQImage-Quality based scoring system CAMERA Mobile High EndCategory based on price YYYY-MMDevice launch date [Device]Device brand and name
000Image Quality score UBER-G SCOREScoring/ranking system name IQImage-Quality based scoring system CAMERA Mobile PremiumCategory based on price YYYY-MMDevice launch date [Device]Device brand and name
000Image Quality score UBER-G SCOREScoring/ranking system name IQImage-Quality based scoring system CAMERA Mobile High Mid RangeCategory based on price YYYY-MMDevice launch date [Device]Device brand and name
000Image Quality score UBER-G SCOREScoring/ranking system name IQImage-Quality based scoring system CAMERA Mobile Low Mid RangeCategory based on price YYYY-MMDevice launch date [Device]Device brand and name

We will the different visuals above to quickly show the most relevant Camera IQ information such as the score, the phone’s name and market position when it was introduced (Year/Month).

Advertising

Why did we create the Uber IQ Camera score?

This scoring system was originally created for internal use. We did it to make better smartphone reviews. From a consumer standpoint, and according to OEMs we talked to, smartphone camera performance is the single most important item when considering a new phone, even ahead of battery life or display quality.

We also have the Uber HW Camera score, which is another proxy for camera performance, based only on camera technical data. It’s great to have such a proxy for just-announced phones, or even for foreign phones we’ve never had our hands on. There is a strong correlation between camera hardware performance and photo quality.

From our editorial review process, we have created the necessary software and algorithms/methodologies to analyze image data and create both cameras scores. It was a lot more work than we bargained for, but it was worth it. Consumers can now search for the best overall camera, the best low-light camera, the best ultrawide camera – and more!

The Uber IQ Camera score pillars

Because numeric scores boil things down, they don’t tell the whole story. To partially remedy this, our IQ score has user-friendly sub-scores, which we consider to be today’s mobile photography “pillars”.

Each covers a use case that everyone is familiar with. By glancing at them, you can understand the nuances much better and choose what’s best for you. When a full camera review is available, you can also read it and see use cases that illustrate the key points of the scores. At the moment those Camera IQ pillars are:

  1. Daylight photo (bright-light)
  2. Night photo (low-light)
  3. Ultrawide photo
  4. Zoom

With our algorithm, we generate a score for each pillar before calculating the final Uber IQ Camera score. The scores are absolute, with no upper limits.

Objective camera tests

There are many objective image-quality camera metrics, and that’s exactly what we are looking at. Objective metrics means that “more” or “less” of them means “better” or “worse”.

Cameras go through a process in which image analysis is based on properties that can be rated as objectively as possible. Rating is separated from personal preferences of “style” of photography (more on styles below). In some cases, software analysis is appropriate, other times, perceptive analysis is preferred.

It is important to point out that any algorithm which computes a score does contain some subjective elements. Image properties that are deemed to be more important may receive a higher consideration, weight, and priority.

In fact, the algorithm is the most important part of any benchmark, as it has more influence on the score than data accuracy fluctuations. For example, and at its simplest, it is like having two algorithms that are “addition” and “multiplication”. Feed them two identical and ultra-accurate pieces of data A and B: A+B and AxB will produce radically different outcomes.

As algorithms get more complex, so is the potential for having vast outcome differences. There is no scientific consensus to rate photo Image Quality, and the photography industry has debated this since photography was invented.

However, you can see the quality of our score algorithm from the quality of its outcome. We have had great feedback from industry-insiders about both the Uber IQ Camera Score principles and its results.

Things such as user interface, filtering styles, and subjective elements are very interesting points but are not be included in the Uber IQ Camera score since they are more related to the camera “user experience” than its image quality. Both are important but are best evaluated separately.

Reality is the anchor. Predictability is satisfaction. Context is everything.

Context Shot, what the scene looks like

Honor View 10 photo

The prospect mobile camera owner (the photo producer) is the first person that Image Quality has to satisfy. We found that there is a very strong correlation between the photographer’s satisfaction and the Camera’s ability to (faithfully) capture what the photographer is looking at. Users want to feel in control.

"USERS WANT TO FEEL IN CONTROL"

When it comes to colors, it is important to separate “quality” from “style”. Since quality can mean many things to many people, we use reality as an anchor and always mean image data quality, when talking about “quality”.

Each lens/sensor pair will induce slight color shifts, which is why pro-photographers will use in-scene measurements to color-correct photos later. It is unreasonable to expect consumers to do such things in their daily lives, so we expect cameras to do a great job of capturing what users see with their eyes.

We refer to a good representation of what our eyes see because it is fundamental to have a good scene intelligence. We don’t expect cameras to be perfect, but this matters – a lot. We will often point out if a camera captures what we see, or that another one might not. Some cameras over-filter images to the point that data is lost, and that could upset photographers for good reason.

Night photography

Context photo: this is how the scene looks like as if you were there

When it comes to low-light photos, that’s often where high-end cameras set themselves apart. It is probably the single most desired feature in mobile cameras. As of late, there has been a trend to artificially “brighten” photos using color filters, which is the easy way to try impressing users. But the truth is: with low light photos, brighter is NOT always better.

Context photo: looking at color spectrum detail

Handset A, looking at the same color spectrum detail

Instead, what we’re interested in is how well the scene’s detail, dynamic range, colors, and light intensity have been captured. Any good night photo can be filtered later to the liking of the user, but an over-filtered night photo can irremediably remove shadows, volumes and contrast and shrink your creative options drastically.

Ultrawide photography

Smartphones have traditionally shipped with a single focal length (~24-26mm) because camera modules are too small to accommodate variable focal length lenses. However, being able to switch to ultrawide (~16mm) is an amazing tool that many compact cameras had for decades.

That’s why we consider ultrawide photography to be one of the necessary pillars and mobile camera with support for ultrawide photography will have an advantage.

Zoom photography

Context Shot, we’re going to zoom on the street art

Every single camera can zoom thanks to digital magnification. Sometimes, people like to zoom for framing purpose or just to take a close-up photo. It’s not always possible to move closer to the subject, so zoom can be useful at times. Having fancy algorithms and optical zoom lenses can be incredibly helpful to get sharper zoomed photos. Zoom is also a fundamental aspect that we look at.

NOT included into Uber IQ Camera (for now), and why

To keep a score to the point and easy to interpret, we wanted to keep its core as straightforward as possible. As such, a number of things were intentionally left out of this first version.

  1. Flash
  2. Ultra-long exposure (Night Mode)
  3. Bokeh
  4. AF speed
  5. Selfie IQ
  6. Video IQ (under consideration)

Flash

A camera can be a powerful tool, but for smartphone cameras, the overwhelming majority of users dislike using a flash, and photos are overwhelmingly taken without a flash.  At the moment, we consider that users have a very small interest in flash performance, so it’s not included as one of our pillars.

Ultra-long exposure (Night Mode/Night Sight)

While ultra-long exposure is an amazing tool, we consider it to be outside of what people think low-light photo performance should be. Long exposure can make up for a small aperture, lens quality or sensor size, but only to a point. However, it’s not clear how often users will accept to wait 4 to 5 seconds to take a picture.

We’ll discuss with users, OEMs, and others who want to join the discussion to look at the usage trends. Our Uber IQ Camera score will evolve over time.

Bokeh (out of focus Blur)

Bokeh is great, but it is also quite subjective. At the moment, we have chosen to leave it out of the score because we were not yet satisfied with the Bokeh scoring options that would have hindered more than helped the IQ rating. We’re open to including it in the future if there’s a strong demand for it.

Autofocus speed

It’s true that AF speed can be important in cases such as action/sports photo or with fast-running kids. However, Autofocus is mostly a succeed/fail element. When successful, it has no importance in regards to image quality. If a camera consistently fails AF, the blurry images will affect the IQ score anyway – speed in itself is a user experience factor, but not a great proxy for image quality.

Selfie camera

Selfies are a completely different use case for a mobile camera. With different hardware, software (filters) and expectations, we do not want to lump selfie IQ into the rear camera IQ – it just didn’t make sense to us, and to a lot of people we talked to.

We discussed building a Selfie score, but in some places, people expect to have a lot of filtering and styling, which makes it very tricky to have an objective score. We’ll keep looking for a good way to build this. For now, perhaps the Uber HW Camera score might be a better proxy.

Video

Video IQ is usually related to the performance of the optics and overall image processing capabilities of the handset. However, we agree that things such as EIS (Electronic Image Stabilization) and details about how 4K or super-slow-mo are captured can be of importance to some users. We’re gathering data on that topic, and we are thinking of having a video score later.

Keep an eye out for new scores!

Thanks for your interest in our Camera benchmark. We really try to provide you with insightful and useful information that will help you get the best camera for your needs and money. Keep in touch via Facebook, Instagram, and Youtube for updates.

Happy Photography!

Note that you can re-sort the table by clicking on any of the headers.

Camera IQ
Smartphone
Day
Night
Zoom
U-wide
Year
227 Huawei Mate 40 Pro 220 257 155 171 2020
224 Samsung Galaxy Note 20 Ultra 227 249 159 154 2020
220 Samsung Galaxy S20 Ultra 225 242 155 154 2020
218 Huawei P40 Pro+ 217 239 157 171 2020
212 Huawei P40 Pro 215 229 150 171 2020
195 Huawei Mate 30 Pro 196 211 118 172 2019
189 Apple iPhone 12 Pro Max 205 207 105 117 2020
181 Xiaomi Mi Note 10 (CC9) 190 193 135 130 2019
180 Huawei P30 Pro 186 196 148 116 2019
178 Samsung Galaxy Note 10 187 194 89 140 2019
178 Samsung Galaxy Note 10+ 187 194 89 140 2019
177 Apple iPhone 11 Pro 189 194 92 117 2019
177 Samsung Galaxy Fold 187 194 89 134 2019
177 Samsung Galaxy S10+ 187 194 89 134 2019
177 Samsung Galaxy S10 187 194 89 134 2019
176 OnePlus 7 Pro 186 194 104 115 2019
175 Samsung Galaxy S10e 187 194 58 134 2019
175 Honor 20 Pro 185 191 103 117 2019
174 Apple iPhone 11 189 194 58 N/A 2019
174 Huawei P30 185 192 103 112 2019
173 LG G8 ThinQ 185 191 57 136 2019
173 Google Pixel 4a 5G 188 182 67 153 2020
173 Google Pixel 5 188 182 67 153 2020
168 Motorola One Zoom 186 178 99 111 2019
167 Huawei Mate 20 Pro 179 179 103 114 2018
164 ZTE Axon 10 Pro 179 171 107 119 2019
162 Samsung Galaxy Note 9 186 193 88 N/A 2018
161 LG V40 173 172 80 129 2018
160 Samsung Galaxy S9+ 184 191 85 N/A 2018
160 Xiaomi Redmi Note 8 Pro 175 176 56 109 2019
158 Samsung Galaxy S9 184 191 59 N/A 2018
157 Google Pixel 4 188 182 86 N/A 2019
155 Apple iPhone XS Max 187 178 87 N/A 2018
155 Apple iPhone XS 187 178 87 N/A 2018
155 Huawei P20 Pro 180 180 103 N/A 2018
154 OnePlus 6T 187 179 59 N/A 2018
152 Google Pixel 3 186 176 62 N/A 2018
151 Xiaomi Mi Mix 3 176 179 82 N/A 2018
150 Honor View 20 179 174 83 N/A 2018
142 LG G7 ThinQ 165 141 57 122 2018
141 OnePlus OnePlus 6 167 167 58 N/A 2018
141 Google Pixel 2 XL 165 168 55 N/A 2017
141 Google Pixel 2 165 168 55 N/A 2017
139 HTC U12+ 180 149 83 N/A 2018
139 Sony Xperia XZ2 181 151 53 N/A 2018
129 Huawei P20 172 136 63 N/A 2018
127 Honor View 10 169 134 54 N/A 2017
126 Honor 10 165 135 53 N/A 2018
124 OnePlus OnePlus 5T 160 136 47 N/A 2017
110 Vivo V11 165 100 53 N/A 2018
109 Honor Honor Play 163 100 53 N/A 2018
87 Huawei Y9 (2018) 142 70 46 N/A 2018
77 Samsung Galaxy J7 (SM-J727U, 2017) 140 50 45 N/A 2017

Filed in Cellphones >Photo-Video. Read more about , , and .

Related Articles on Ubergizmo