Wednesday, March 29, 2017

What sets a slice of pizza apart from others?

There are two options within walking distance in my very small college town when it comes to pizza: New York Pizzeria (known to most people as Slices), and Oliveri's. Slices stays open into the late night, catering to many last-minute crammers and drunk college kids. Oliveri's draws from a slightly different niche: fewer intoxicated consumers, and probably more families or groups of people eating together earlier during the day.

I've eaten an estimated combined total of 100 slices from these two places over my four years at college, which I don't think is a lot. Most of my pizza consumption over the past four years at college has come from Slices after nights out, as well as pizza from either place during brown bags, events, or meetings. Almost all 100 slices consumed were plain cheese slices, since that is Slices' specialty as well as a generic choice for catering.

For a while, I've wondered if I could differentiate between pizza from both places in a blind taste test with a similar format to a that of Buzzfeed video. If you asked me, I would say I prefer Slices, but I think that's solely because of my memorable moments there and my lack thereof at Oliveri's. I don't have a particularly refined palate, and I'm not sure I actually know the difference between the two.


Every time I mentioned this idea, I would get pretty strong reactions, usually along the lines of "they're so different, you're so dumb, why are you even bringing this up".

None of my friends seemed willing to help me administer a blind taste test, so I got new friends I took matters into my own hands. The best way to procrastinate working on my thesis, I decided, was to work on another independent project. I planned a blind taste test for participants to see how many of them would be able to correctly distinguish between pizza from the two places.

Main Objectives of Project
1. See how many people correctly distinguish between Slices and Oliveri's solely by taste.
2. Look for possible statistical relationships between people's consumption of pizza from these two places and their ability to correctly distinguish between Slices/Oliveri's.
3. Look for possible statistical relationships between people's confidence in being able to distinguish pizzas and people's actual ability in distinguishing pizzas
4. Identify patterns in what people think distinguishes pizza from these two places, since I clearly don't know

Hypotheses
1. People's accuracy rate in distinguishing between pizzas is higher than 50%, since that would be the accuracy rate from randomly guessing. If you're told that you ate two different pizzas and you have to guess which one you ate first and which one you ate second, you will either be 100% correct or 100% wrong. The people I've spoken to (which I admit may or may not be a representative sample) seem very assured that the pizzas are clearly different, so I would expect a higher accuracy rate than 50%. I would also imagine that this study already attracts a population that is somewhat interested in pizza, thus increasing the likelihood that people's ability to distinguish between the two is higher than random chance.

2. People who consume pizza from these two places frequently are more likely to correctly distinguish between the two pizzas compared to those who do not consume pizza as frequently.

3. People who are confident that they can distinguish between the two pizzas are more likely to correctly do so, compared to people who are not.

Methods
Before
A day before the study, I put up a Facebook post calling for participants. "Like this status for hype, and love this status if you'd like to participate!" I wrote. Since I was concerned about the low number of "loves", I used my social capital and powers of persuasion to convince some of the people who had initially only "liked" it to also join. In total, I managed to recruit 23 participants, all of whom have consumed pizza from both places before and are not deathly allergic to cheese pizza.

I ordered two large plain cheese pizzas - one from Slices and one from Oliveri's. I do not drive, so I got a friend to drive me downtown to pick up the pizzas, thus maximizing freshness.

The man at Oliveri's gave me strange looks as I stashed all their available plastic forks away into a plastic bag and, upon being told that I would have to wait a few minutes, scurried across the street to pick up my pizza from Slices. He probed into my behavior. After I explained my plan, he gave me a business card and told me to share the results of my study with them. So if you're reading this, hello!

In order to eliminate the possibility of self-confusion between the pizzas, I brought a big bold marker with me - at Slices, I wrote SLICES on the box as soon as I got it, and you get the idea. After I returned to my humble abode/the location of the study, I cut up one pizza entirely into pieces and placed them on one orange plate before opening the other pizza box. I also privately noted down the "correct answers" on my phone before starting (Oliveri's on the orange plate, Slices on the white plate).

I told participants to show up at my house whenever they wanted during a designated two-hour window of time.

During
This study consisted of three parts: the pre-tasting form, the actual blind taste test, and the post-tasting form.

The pre-tasting form had two parts: a consent form with detailed instructions (I don't know why people laugh when I tell them that I actually typed up a consent form - this is serious business), and a brief questionnaire into:
a) how often participants ate pizza from each of the two places
b) which pizza participants preferred, and why
c) whether participants felt that they would be able to distinguish between the two pizzas in a blind taste test, and why

The instructions were fairly straightforward; I explained the setup of my experiment and my intention to apply the honor code in trusting that participants would keep their eyes closed during the taste test. I felt that this was preferable to blindfolding participants because it would be unsanitary to make multiple participants put the upper half of their faces - a place where pimples are bred - on the same piece of cloth.

I was asked several times if my study was a trick and whether I would actually be serving the same pizza twice. I emphasized that this was not the case.

During the taste test, participants had to rely solely on their sense of taste in distinguishing between the two pizzas. I endeavored to minimize any confounding variables by asking them to close their eyes during the experiment. They were given plastic forks, and I gently guided their hands into picking up pieces of pizza from my plates, one at a time, with their forks. This thus took away the possibility that participants could distinguish between pizzas by their appearance, or by the way they felt by touch.

When there were multiple participants at once (peak times for attending my study were during the start of each hour, or half past), I had them sit facing away from each other.

After participants had eaten both pieces of pizza and opened their eyes, I had them fill out the post-tasting form, which covered things like:
d) which pizza did they eat first, and which pizza did they eat second?
e) were they confident about their answer, and why?

Participants had a unique number on their pre- and post-tasting forms (01-23), and this was done with the hopes of getting more insights into individual data points.

After
At the end of the second hour, my thesis was untouched, all the pizza had been eaten, and I decided I didn't want to see, touch, or eat a slice of pizza in the next month. I washed the plates, put everything away, and - with great excitement - entered all the answers from the pre- and post-tasting forms into a spreadsheet. I ate pizza two days later. 

Disclaimer
23 data points is insufficient for making conclusions, this is at best a pseudo-study, etc. Also completely self-funded and not sponsored by either pizza place/any academic departments at my college.

Results
Hypothesis #1: People's accuracy rate in distinguishing between pizzas is higher than 50%.

14 people were correct
9 people were incorrect

At an accuracy rate of 61%, I am not convinced that my participants did significantly better than randomly guessing. According to this online calculator, a 61% accuracy rate among 23 people is not statistically significantly higher than a 50% accuracy rate.

There were no "strange" responses - none of the 23 participants wrote that they ate two of the same pizza.

Hypothesis #2: People who consume pizza from these two places frequently are more likely to correctly distinguish between the two pizzas.

On the pre-tasting form, people's options for describing how often they ate pizza from each place were:
a) Never/rarely
b) Fewer than once a week
c) Once a week
d) 2-3 times a week
e) More than 3 times a week

In order to quantify these answers, I assigned the following numerical values to these answers:
a) 0
b) 0.5
c) 1
d) 2.5
e) 3.5

The people who correctly distinguished between the two pizzas ate pizza from Slices an average of 0.57 times a week, and pizza from Oliveri's an average of 0.46 times a week.

The people who did not correctly distinguish between the two pizzas ate pizza from Slices an average of 0.55 times a week, and pizza from Oliveri's an average of 0.5 times a week.

Again, there does not appear to be a clear relationship between the frequency of pizza consumption and the ability to correctly distinguish between pizzas.

I will also add that people's frequency of pizza consumption surprised me a little; I assumed the average person consumed it more frequently than the answers suggested. But perhaps my 23 participants do not accurately reflect the average student at my college? None of the 23 participants eat pizza from either place more than 3 times a week, and only one person chose the "2-3 times a week" option at all.

Hypothesis #3: People who are confident that they can correctly distinguish between the two pizzas are more likely to correctly do so.

This seems intuitive, right? Participants' confidence was measured in a yes/no answer, and participants had to state their confidence both before and after the blind taste test. Given that the yes/no answer is binary, I assigned a 1/0 value to these responses.

Because of the unique number that was assigned to each participant, I had insights into each individual's set of responses. There are four possible outcomes when it comes to confidence in my survey:
A) confident before tasting, confident after tasting
B) not confident before tasting, confident after tasting
C) confident before tasting, not confident after tasting
D) not confident before tasting, not confident after tasting



Among the people who got it right, 71% (10/14) of them were confident before tasting and 79% (11/14) of them were confident after tasting. It is interesting to note that the increase between the pre- and post-confidence levels was not just a result of one person becoming confident after tasting: instead, this increase is because four people were in category B, and three people were in category C.



Among the people who got it wrong, 56% (5/9) of them were confident before tasting and - interestingly enough - 89% (8/9) of them were confident about their answers after tasting. The one person in this group who was not confident after tasting was the only person out of all 23 participants who was in category D.



Additional observation
A total of 11 people said they prefer Slices, 5 prefer Oliveri's, and 7 have no preference. All 5 people who said they prefer Oliveri's correctly distinguished between the two pizzas. Coincidence? Or are Oliveri's fans just more skilled at this?




So, what sets a slice of pizza apart from others?
The participants were asked, both before and after tasting, to - if they were confident - explain the distinguishing factors between the two pizzas.

The funny thing is that the content of answers provided by the people who got it right were very similar to the answers from people who got it wrong: people pointed out contradictory differences in the dough, the cheese, and the sauce.

Here are some amusing direct quotes from the people who got it wrong:
1. (From somebody who said they preferred Slices): "The first pizza tasted better - it must be Slices"
2. "The cheese flavor of the first piece was more distinct and memorable, reminds me of nights out"
3. "Turns out I do know what Slices tastes like"
4. "You can just tell from the texture"
5. "The textures are different, the tastes are different"
6. "As a New Yorker, I'm a pizza expert. The sauce on the first pizza is sweeter, and the crust is crispier instead of chewy"

Key takeaways from this study
1. I need a new hobby
2. People are not really that good at distinguishing between the two pizzas
3. There appears to be no solid relationship between frequency of pizza consumption and ability to distinguish between the two pizzas
4. There appears to be no solid relationship between confidence and ability to distinguish between the two pizzas
5. There are (probably) many people on this campus who have a preference for one pizza over another, but this preference is (unknowingly) rooted in other factors, including atmosphere and personal memories associated with each place
6. This study would be a lot cooler/sadder (depending on who you ask) if there were hundreds or thousands of participants

Acknowledgements
This study was made possible by the participation of 23 students, the patience of my housemates, $26, and a shitty thesis.