Melatonin: Preliminary Results

I now have roughly 2 months of sleep data. The first month (the control phase) I didn't take any sleep supplements, but in the second month (the experiment phase) I took exactly 3mg of melatonin approximately 30 minutes before going to bed each 'night'. During the control phase, I got an average of 406 minutes of sleep per night, with a standard deviation of 136 minutes.

In the experiment phase, I averaged 444 minutes of sleep per night, with a standard deviation of 98 minutes.

Therefore, on melatonin, I average an extra 38 minutes of sleep per night and experienced significantly less variation in the total length of sleep.

For the control phase of the trial, I averaged 115 minutes in REM sleep, with a standard deviation of 43 minutes.

During the experiment phase, I averaged about 125 minutes of REM sleep, also with a standard deviation of 41 minutes.

Therefore, while on melatonin, I was getting about 10 extra minutes of REM sleep per night with about same amount of variance.

For the control portion of the trial, I got about 37 minutes of deep sleep on average, with a standard deviation of approximately 23 minutes.

In the experiment phase, I got an average of 34 minutes of deep sleep, with a standard deviation of 12 minutes.

While taking melatonin, I average 3 minutes less deep sleep, but with quite a bit less variance as well.

During the control phase, I woke up about 3 times per night, with a standard deviation of 2 times.

In the experiment portion, I also awoke about 3 times a night with a standard deviation of 2 times.

Being on melatonin didn't seem to make any difference in the number of times I woke up during the night.

According to the preliminary data, while on melatonin, I seemed to get more total sleep, more REM sleep, less deep sleep, and wake up about the same number of times each night. Because this isn't enough data to be very confident in the results, I plan on continuing this experiment for at least another 4 months (2 on and 2 off of melatonin) and will analyze the results for the significance and magnitude of the effects (if there really are any) while throwing out the outliers (since my sleep schedule is so erratic).

The Paleo Diet: Follow-Up V

Okay, I finally got around to digitizing all of my secondary data (from my doctor's records, lab tests, and United Blood Services) relevant to the efficacy of my dietary experiments and merged it with my primary data (the data I collected myself), so I am now in a much better position to make an final appraisal than previously was the case. Also, thanks to my recently acquired programming skills (although still rudimentary at this point) I was able to reorganize my data so that all of the points share the same date format, which should make it much easier to visualize the passage of time (rather than merely the passage of measurements) in my graphs than before.

Firstly, did I lose weight? I already had a high degree of belief that this was the case (from clothing fit and pictures taken during this time period), but since I started recording my weight daily simultaneous with beginning the diet, my assessment lacked the degree of objectivity I think appropriate for these kinds of things. Here is a graph including all of the information about my weight that was available to me at the time of this post (which includes datapoints further in the past than my original analysis):

As you can see from the graph (the "lumpiness" in the first half is due to the relative poverty of information I have about myself from before I became a self-tracker), I was slowly gaining weight during the year leading up to the start of the experiment. The weight loss I did experience started immediately upon starting the diet and continued at a fairly rapid rate for months before slowing down and then oscillating between values that the Singapore Government's Health Promotion Board deems a "healthy range".

Secondly, did I maintain my strength and endurance? When I started my diet I went to the gym 4-6 days a week, alternating between running and weight-lifting. I currently go to the gym 4-6 days a week, alternating between running and weight-lifting. All of my lifts have increased in weight while keeping sets, reps, and rest periods constant. Also, I now run at a higher rate of speed for the same amount of time (and therefore, am running a greater distance). 

Thirdly, has my health suffered? Subjectively, this is an obvious no. My body feels better along almost every imaginable dimension. On the other hand, the placebo effect can be quite powerful and the human capacity for self-deception almost limitless. Since the most common argument against the efficacy of the Paleo Diet for long-term health is that high-fat diets increase the chance of getting heart disease and stroke (although, bacterial infection and calcium deficiency are also fair worries), I have decided to concentrate on measures of cardiac health.

The American Heart Association recommends a systolic (maximum) blood pressure of between 90-119 mmHg and a diastolic (minimum) of between 60-79 mmHg. Blood pressure can very quite a bit throughout the day and between days, so the average is more important than any individual measurement. Before starting the diet my average systolic blood pressure was 115 mmHg, but the sample includes only 4 datapoints which isn't enough to know with much certainty what the actual value was during this time period . In any case, the average from after starting the diet until now (including 119 datapoints, which allows us to be fairly confident in the result) is 112.6 mmHg.

Pre-Paleo my diastolic blood pressure came out to an average of 64.8 mmHg, while afterward it ended up being 61.8 mmHg.

As is apparent from the graphical representations of the data, my blood pressure started out in the AHA's "desired" category and ended up in the same category. Interestingly, in both cases, my average blood pressure decreased after starting the diet.

Cholesterol is generally considered (by the medical profession) to be a measure of heart health, but Paleo advocates disagree. The following graph shows my total cholesterol levels before and after going on the Paleo Diet:

My average total cholesterol started out at 154.2 mg/dL, but is now 201.3 mg/dL. This appears to be an unfavorable change (assuming the consensus in the medical field is correct), but only barely puts me over the edge between "desirable" and "borderline high". While perhaps worrying in isolation, in conjunction with the fact that my post-experiment ratio of total cholesterol to HDL cholesterol is 2.9:1 (which easily falls into the AHA's recommendation of "less than 5:1") it doesn't seem to be a serious threat to my cardiovascular health.

In short, I lost a significant amount of weight, increased my strength and endurance, and probably didn't endanger my health in any other way. This is essentially the same as my initial appraisal, but my confidence in this result is now significantly greater due to the much larger quantity of data and the extended time period (incorporating "new" data from both before and after my original post) used in this later analysis. I am now comfortable making the claim that if you are like me in relevant ways (especially if you are closely related to me), then you should worry a little less about eating large amounts of fat and worry a little more about eating things high in processed carbohydrates.

A (Not So) Random Selection of Population Statistics

Sleep: The Baseline

I was finally able to procure an EEG device thanks to some generous benefactors*. The following summary parameters are based on data (21 datapoints total) collected using the Zeo headband and a Samsung Galaxy S running Android 2.3 between 06/10/2012 and 07/03/2012:

Total Time Spent Asleep
Mean: 387 minutes
Median: 450 minutes
Mode: 471 minutes
Range: 506 minutes
Minimum: 18 minutes
Maximum: 524 minutes
Sum: 8,125 minutes

Time Elapsed Before Falling Asleep
Mean: 18 minutes
Median: 18 minutes
Mode: 13 minutes
Range: 38 minutes
Minimum: 1 minutes
Maximum: 39 minutes
Sum: 377 minutes

Time Spent Awake
Mean: 10 minutes
Standard Error: 3
Median: 6 minutes
Mode: 0 minutes
Range: 57 minutes
Minimum: 0 minutes
Maximum: 57 minutes
Sum: 217 minutes

Time Spent in REM Sleep
Mean: 108 minutes
Median: 125 minutes
Mode: 134 minutes
Range: 154 minutes
Minimum: 5 minutes
Maximum: 159 minutes
Sum: 2,274 minutes

Time Spent in Light Sleep
Mean: 245 minutes
Median: 290 minutes
Mode: 290 minutes
Range: 320 minutes
Minimum: 13 minutes
Maximum: 333 minutes
Sum: 5,153 minutes

Time Spent in Deep Sleep
Mean: 34 minutes
Median: 28 minutes
Mode: 24 minutes
Range: 104 minutes
Minimum: 0 minutes
Maximum: 104 minutes
Sum: 708 minutes

Number of Awakenings
Mean: 3 times
Median: 3 times
Mode: 5 times
Range: 9 times
Minimum: 0 times
Maximum: 9 times
Sum: 71 times

After only a few days, I realized that I was getting much less sleep than I thought (less than 6:30 per "night") and decided to make some adjustments. I set my alarm for an hour later and purchased one of those cheap "sleep masks" from Walmart. I don't anticipate making any other large changes before starting any of my sleep experiments, so this set of data will act as something of a baseline with which to compare later data to. My first experimental intervention (having to do with sleep) will be melatonin. I plan on taking it on-and-off for at least 6 months (possibly up to a year, if that is what will be required to reach statistical significance) in order to measure its effects and will perform a cost-benefit analysis on melatonin supplementation shortly after completing the trial.

*Thanks mom and dad for the birthday present!

(In)sanity: 300 Datapoints

According to the Stanford Encyclopedia of Philosophy, "a credence function is actually calibrated at a particular possible world if the credence it assigns to a proposition matches the relative frequencies with which propositions of that kind are true at that world". Now, unless you are really into philosophy or statistics, this probably won't mean a whole lot to you. Luckily, Yvain has an easy to grasp explanation of this concept:

The rationality literature has especially focused on one particular subjective mental estimate: our feelings of probability. For example, someone may say they feel 80% certain that Germany is larger than France. However, if they consistently answer questions like this with 80% confidence, and only get 60% right, then we say they are mis-calibrated: their subjective mental estimate of probability has a consistent mismatch with a more normatively correct probability. Calibration means revising your subjective mental estimate until it matches the objective value it tries to estimate; so that when you estimate something with 80% confidence, you get it right 80% of the time.

Now, it seems to me that much of the time when we talk about someone or another being insane or crazy, we are, in effect, saying that that person is extremely poorly calibrated (at least, when it comes to that particular topic). For instance, imagine a person named Alice. Alice has a problem going out at night because she assigns 95% chance to the proposition that she will be abducted by aliens and transported to Alpha Centauri. Yet, each time she does go out, this event does not occur. If she continues to assign the same probability next time, she is failing to calibrate her beliefs correctly and is likely to be considered insane because of it.

As it turns out, calibration can actually be measured by having the subject make predictions about verifiable or falsifiable events based on their beliefs and then scoring those predictions as the events do or do not occur. In a very real sense, this is a way to measure someone's degree of sanity. Ever since I read Gwern's article about PredictionBook I have been participating in such an experiment myself. I even went so far as to publish a short article encouraging others to do the same. So far I have made over 300 predictions that have been scored and probably over 1,000 in total.

My predictions have ranged over extremely silly and serious topics including whether the new season of My Little Pony with start with a 2-part episode, if I will obtain a particular IT certification within a certain timeframe, who the next POTUS will be, and whether or not George Zimmerman will be convicted of murder. The following is a graphical representation of the current state of my (mis)calibration:

The cyan line indicates how a perfectly-calibrated agent (there almost certainly aren't any humans who meet this standard) would have assigned its probabilities and the other line are the actual probabilities I did assign over the 300+ datapoints I have already collected. A very sane person's line (in a given domain) should more-or-less track the straight line going from 50-100. I'll let the reader decide for themselves if I am meeting this standard, but I think I did pretty well for not having much formal training. On the other hand, if we plotted Alice's data and had her make predictions about the behavior of extraterrestrials, then we should expect the line of her actual probability assignments to deviate wildly from the line of perfect calibration (perhaps, even having a downward slope). Although, to be fair to Alice, she most likely is pretty sane when it come to beliefs about what kinds of things are edible or poisonous (otherwise she would no longer be with us) and if we tracked the implicit predictions of political pundits on CNN on the topic of politics, they would probably come out looking quite insane (because politics is the mind-killer).

I plan to continue this experiment to determine if I can improve my calibration over time and will post on this topic again once I reach approximately 600 datapoints. Until next time...stay sane.

The Paleo Diet: Follow-Up IV

According to Dr. Ravi Bhatia, a doctor who specializes in leukemia, you shouldn't experiment on yourself, because "you won't learn from it, and others won't learn from it, either." While this view is almost certainly too strongly worded, there are many ways self-experimentation can go wrong. Besides being potentially dangerous, it is far too easy to fall prey to myriad cognitive biases. Of course, scientists, working in their labs, face all of these same issues, but perhaps not to the same extent. After all, it is common wisdom that it is much easier to be objective about others than ourselves.

In order to combat the pitfalls specific to self-experimentation, it helps to have multiple sources of independently collected data. As I mentioned before, United Blood Services records several pieces of information about each and every person that chooses to donate blood and provides that information to donors via their official website. Luckily, I donated blood both before and after starting my Paleo Diet experiment, so I do have some data in which to check against that collected by myself on at least several different dimensions.

I started the experiment almost immediately after my blood donation on 10/12/2011, so this graph represents 6 independently collected datapoints of my total cholesterol (5 before starting the experiment and 1 after). For some reason that escapes me, my donation on 7/23/2010 only has data for total cholesterol and not any other of the measures. Even simply eyeballing the graph, it is quite clear that my total cholesterol has risen since I started eating Paleo. The mean pre-experiment total cholesterol is 154 mg/dL, which is a full 47 mg/dL lower than the datapoint from after I started the experiment. WebMD tells me that my pre-experiment total cholesterol is "desirable", but that my during-experiment level is "mildly high". On the other hand, my total cholesterol level is only in the "mildly high" category by 1 mg/dL which is probably within the margin of error. In any case, I think it is (moderately) safe to conclude that adopting a Paleo Diet has caused my total cholesterol to increase (but not dangerously so).

Based on the 3rd-party data, my blood pressure hasn't seemed to change much since I started the experiment. The pre-experiment mean for my systolic blood pressure was 115 mmHg and my during-experiment datapoint was 118 mmHg. The difference is well within the margin of error. My previous diastolic blood pressure had a mean of 65 mmHg while my latest datapoint was 75 mmHg. I'm not really sure if this falls within the margin of error, but I regularly observe about that much variance between two consecutive days according to the data I have been collecting on my own. A possible confounding factor with the particular set of data is that on the pre-experiment days I donated blood shortly after waking up, while the latest datapoint was just before I went to bed (due to shift changes at work). I'm not sure how large of an effect (if any) this has had on the data, but either way, I don't think my blood pressure has changed much because of my diet.

My mean pulse pre-experiment was 65 BPM and my during-experiment datapoint was 53 BPM. While lower numbers are usually considered to be healthier, numbers lower than 60 BPM are considered to be a condition called bradycardia. WebMB advises me that without the presence of negative symptoms like light-headedness or chest pain, "a slow heart rate is sometimes normal and can be a sign of being very fit" and that "healthy young adults and athletes often have heart rates of less than 60 BPM". This doesn't seem to be much cause for concern and is weak evidence that my physical fitness has improved.

Hemoglobin levels are something that United Blood Services records, but that I don't. Judging from the graph, it appears that my hemoglobin levels had been increasing over time and then reversed after starting the experiment. I basically have no idea what this means or if this is even enough data to be meaningful. If anyone knows how to interpret this information I would very much like to hear it.

Body temperature is another measure I haven't been recording, but that UBS has. I was slightly colder during my last visit than on previous occasions. The mean pre-experiment temperature was 98.4 degrees Fahrenheit, but my most recent datapoint was 97.3 degrees Fahrenheit. I suspect this is another one that has been confounded by me switching the time of day of my visits. On the other hand, I think I would have predicted an increase in temperature and not a decrease, because only for the last datapoint was I completely "warmed up" by going about an entire workday before donating blood. I'm not sure what to make of this. Any ideas?

According to the graph of the UBS data, my BMI was slowly increasing and then dropped off significantly. My pre-experiment BMI came out to a mean of 25.9, but my latest datapoint put me at 21.6.

Overall, the data collected by UBS seems to roughly line-up with the data I collected myself. This increases my confidence that the relationships I have been observing in my data aren't merely artifacts and that I'm probably not too hopelessly biased to use the results I get from self-experimentation for important life-style choices.

The Paleo Diet: Follow-Up III

My weight seems to have flat-lined within the optimal BMI range:

I have now collected enough data to be relatively sure that my blood pressure and resting heart rate are within healthy ranges:

My average diastolic blood pressure over the last ~3 weeks was 110.9 mmHg. The cutoff point for prehypertension is 120 mmHg. Only two of my data points were on or over this limit. Therefore, on average my diastolic blood pressure is 9.1 mmHg below the dangerous range. Over the same time span, my diastolic blood pressure averaged 61.8 mmHg. This is 18.2 mmHg less than the cutoff for prehypertension (which is itself much lower than the cutoff for hypertension). Not a single data point exceeded the recommended limit. Also, my average resting heart rate was 55.2 BPM. This puts me almost precisely at the median for adults that are well-trained athletes.

So far, the balance of evidence leans towards the Paleo Diet being both a good way to lose excess fat and relatively safe for the cardiovascular system. Of course, my evaluation is open to revision once more data comes in. In order to collect more evidence as to the effects of the diet on heart health, I plan on getting blood tests monthly starting next month.

The Paleo Diet: Follow-Up II

Although I hadn't been getting regular blood tests before I started my experiment with the Paleo Diet, I am a semi-regular blood donor. Apparently, United Blood Services both records the cholesterol levels of their donors and uploads these records to their website. Unfortunately, I only have access to the data for total cholesterol and not HDL, LDL, or triglycerides. The data are as follows:

The first 3 data points are pre-experiment, while the last datum is from during the experiment. Therefore, my total cholesterol has almost certainly increased since starting the diet.

Here are the latest data on my weight, which seems to be stabilizing:

The N-back Experiment: Analysis

N-back is cognitive exercise developed by Wayne Kirchner in 1958 in order to test working memory. In 2008, PNAS published a research paper by Jaeggi, Buschkuehl, Jonides, and Perrig that concluded that prolonged use of n-back improved fluid intelligence. Shortly thereafter, the scientific journal Intelligence published a research paper by Moody that called into question the methodology used in the former study. So, does practicing n-back make you smarter? Let's find out.

In order to see if n-backing makes me any smarter, I'll need a reliable way to figure out how smart I am both before using n-back and after. The obvious solution is the IQ test. IQ is far from being a perfect measure of intelligence, but it is the most widely used test by psychometricians (suggesting that there aren't any better alternatives). Lucky for me there are several that are available for free on the Internet.

Unfortunately, like most tests, IQ is subject to the practice effect. What this means is that even if the intelligence of the test-taker doesn't increase, they may score higher on a later test for the simple reason that they have practice taking that particular test under the same conditions. To avoid this I could either take the test over and over again until the practice effect stops increasing my score or space the testing out far enough so I lose familiarity with it. For this experiment I will utilize the later strategy and only take a particular test 2 times (total) during the entire experiment. Because the practice effect cannot be completely eliminated, I will only consider an increase of equal to or greater than 5 IQ points to be an increase in my intelligence (rather than simply my performance on a given test). It should be noted that this value is simply an educated (barely) guess and is not the result of some kind of learning curve calculation.

For the purposes of this experiment, I chose a test based on the Raven's Progressive Matrices used by the Mensa International chapter in Denmark. The exact test can be found here.

• Pre-Experiment IQ: 118

Starting 04/01/2013, I will start a n-back regimen of 20 minutes per day (in one continuous session), everyday. I will continue for 60 days and then retest my IQ and post the results. The precise platform I will be utilizing is the (free) Brain N-Back application for the Android operating system running on my Samsung Galaxy S smart phone.

I don't have very high hopes that the results will measure up. After all, people have been trying to increase their IQ for decades (at least) with little success and the original study has yet to be replicated by other researchers. I estimated the probability it will work here. I'll post an appraisal of the objective data I collect as well as my subjective impression a short time after completing the experiment.

The Paleo Diet: Follow-Up I

One of the primary criticisms of the Paleo Diet is that high-fat diets lead to the development of cardiovascular disease, so it seemed especially prudent to track measures of cardiac health. Here are my most recent data over the last 2 weeks on systolic blood pressure, diastolic blood pressure, and resting pulse rate:

The Paleo Diet

Proposition

The Paleo Diet supposedly mimics the diet of hunter-gatherers from the Paleolithic era (which ended about 10,000 years ago). Intuitively, it includes foods that can be hunted (meat, fish, etc...) or gathered (eggs, fruit, vegetables, etc...), but not those that can't (grains, legumes, sugar, etc...). In practice, this usually means eating a large amount of fat and protein, but not very many carbohydrates. In this sense, it is a subset of low-carb diets (but this is not true of all versions of the Paleo Diet). The proponents of the Paleo Diet explain its efficacy by resorting to evolutionary explanations, which some dietitians and anthropologists dispute. I will skip over the theoretical debate completely and only focus the truth value of the following proposition:  

The Paleo Diet will work (for me).

What does it mean for a diet to work? Time to operationalize. For the purposes of my self-experiment, I will consider a diet to work if and only if, while on it:

1. I lose a significant amount of fat. In this case, 10 lbs seems reasonable.

2. I maintain my strength and endurance. If I am unable to continue my weekly weightlifting regimen at my current weight levels, I will consider this condition not to be met. Also, if I am unable to continue to run the same distance in a given amount of time, then the diet will not count as working.

3. I don't otherwise become unhealthy. More precisely, this would require that I don't get any abnormal values on the biochemical analysis (blood test) included in a physical examination at my doctors office.

Prior Belief

There are several reasons for doubting whether any diet will work for me, let alone a high-fat diet:

1. I'm in a reference class in which 66% of the people develop obesity before reaching 60 years old. It already isn't looking good.

2. According to my back-of-the-envelope calculations, I will be eating much more than the Food and Drug Administration's daily reference values (DRV) for saturated fatty acids, cholesterol, and probably protein. Also, I will be eating more than 90% less than the DRV for carbohydrates. If you think the FDA has any idea what they are talking about, then this should be worrying.

3. Since I have the CC version of the rs1801282 single-nucleotide polymorphism (SNP), 23andme's health analysis software advises me that "a diet high in monounsaturated fat is not likely to have beneficial effects on BMI or waist circumference". Also, I have the AG version of the rs662799 SNP, so "dietary fat consumption is not associated with changes in BMI". In other words, I don't have any information about my genetics that would lead me to think that this is going to be a particularly effective diet.

Evidence

According to the data, I lost approximately 30 lbs since starting the Paleo Diet. This brings me from a not-so-great BMI of 26.5 to a much safer BMI of 21.5. The Singapore Government's Health Promotion Board claims that a BMI in the range of 23-27.4 poses a "moderate risk of developing heart disease, high blood pressure, stroke, diabetes", but that a BMI in the range of 18-22.9 constitutes a "low risk (healthy range)". This easily beats my requirement of losing 10 lbs in order to claim that the Paleo Diet works.

I continued my regular weight-lifting routine (more or less) through the entire time I was on the diet and have increased my weight on both the back squat and deadlift. My bench press stagnated, but didn't get any worse. Also, I am now running a greater distance in the same amount of time as I was before starting the diet. This means that 2 of my 4 fitness goals and predictions have already been confirmed on PredictionBook (squat and deadlift, but not bench press or running).

I did finally go in for my yearly (actually less) physical exam. I should note that I did not fast before my blood test (which the doctor didn't think warranted me losing sleep in order to do, but that the lab tech seemed to think was important). My cholesterol is somewhat high, but is still within the typical (healthy) range. Over the phone, my doctor advised me that my cholesterol was on the high end of the normal (healthy) range, but it appears my actual lab results (of which I just obtained a copy) are showing a "flag" on my total cholesterol. Here are my actual results:

• Total cholesterol: 207 mg/dL
• Triglycerides: 102 mg/dL
• HDL cholesterol: 72 mg/dL
• LDL cholesterol: 115 mg/dL

According to the American Heart Association (AHA), a total cholesterol between 200-239 mg/dL is considered "borderline high" and puts you at higher risk of coronary heart disease than for values less than 200 mg/dL, but much less so than for values over 240 mg/dL. The AHA thinks my LDL cholesterol is "near or above optimal", so there isn't much to worry about there. Also, my triglyceride level is "normal".

On the other hand, the AHA also says that more important than the total number is the ratio of total cholesterol to HDL cholesterol. They recommend keeping the ratio under 5:1 and striving for 3.5:1. My ratio comes out to 2.9:1, which is even lower than what you are supposed to strive for! I'm not exactly sure how to interpret this result. This does put some doubt as to whether the diet has successfully met my initial success conditions, but it certainly isn't an outright falsification. Of course, advocates of the Paleo Diet usually claim that the common understanding of cholesterol by medical practitioners is simply wrong. I don't have the relevant background knowledge to evaluate their arguments for myself, so I assume that mainstream medicine is correct and will count this as a minor point against the efficacy of the diet.

Blood tests aren't the only indicator of potential heart problems. Blood pressure is another. I tracked my blood pressure and pulse over the last six days and the result is less ambiguous than the blood test. The AHA recommends a systolic blood pressure of less than 120 mmHg and a diastolic blood pressure of less than 80 mmHg.

My blood pressure readings were consistently within the healthy ranges set by the AHA.

Posterior Belief

The experiment provides some confirmation of the proposition, but is somewhat ambiguous due to not completely fulfilling the third condition. I plan on continuing with a (slightly modified) version of my current diet pending further testing.

Part 2 | Part 3 | Part 4