Wednesday, 10 February 2016

Industry Insights

The editor-in chief at Public Health Nutrition makes a backhanded case for including industry as referees on academic papers.

As recap: that journal accepted a paper last June titled "Ultra-processed foods have the worst nutrient profile, yet they are the most available packaged products in a sample of New Zealand supermarkets".

Katherine Rich, of the Food & Grocery Council, wrote a letter to the editors of the journal noting substantial methodological problems with the paper. Most importantly, the authors did not seem to understand how the Nutrient Profiling Scoring Criteria works. Breakfast cereals, for example, are poorly treated in the NPSC data: a cereal that is, say, 50% fibre would show up as being 4.7% fibre. Much of the content is then missing in that data. Rich demonstrates how this would bias their figures.

The published letter is far more in-depth than most referee reports I've seen. Rich goes beyond identifying a potential problem to show exactly how that problem biases the results presented. It is not common to have a journal referee replicate part of your work, fix it, and show you the consequences of what you've messed up.

The journal then contacted the Chief Public Health Nutrition Officer at Food Standards Australia New Zealand to check whether Rich were right. And the editorial on it all says:
The subsequent submission of a letter from Dr Mackerras, the Chief Public Health Nutrition Officer at Food Standards Australia New Zealand (FSANZ), reiterated some of the same methodological concerns(15) and confirmed the value of addressing those concerns in detail.
The editorial notes that had Rich submitted her findings as a peer-reviewed article rather than as a letter, it likely would have "received closer scrutiny or been met with greater scepticism" than one received by an author without an obvious conflict of interest.

The editor goes on to try to justify one-sided scepticism, noting cases where industry has tried to skew things, and impugning Rich's character. But it seems a bit of an odd turn where Rich pointed out a problem that only an industry expert would have caught. Rather than thanking her for correcting errors, they highlight that Rich plays an advocacy role and criticise her for having published her critique on the FGC's website. Nowhere are the academics who wrote the paper criticised for not having checked into their data source.
As a side note, we were concerned that the methodological issues raised by Ms Rich and Ms Mackerras from FSANZ should have been caught during the review process and perhaps were missed because the findings confirmed what the reviewers hoped or expected. In this case, however, the comments raised by the three reviewers appeared to be quite balanced overall. The points raised by Ms Rich and Ms Mackerras were perhaps only obvious to people who were very familiar with and invested in the scoring system used in the paper. Our concern about ‘confirmation bias’ in the review process relates to the possible role of undisclosed, ideological interests – a second pitfall of disclosure statements.
I don't know how many times I've heard public health activists say that industry deserves no place at the table. But leaving them out systematically biases things. You mess things up when you deliberately leave out people who know things that you don't.

Imagine if journal editors considered including as referees industry folks who actually know what's going on in their area. The Editor always can ditch silly suggestions that come of it, and the industry referees would know that. But if an author's doing violence to the data in ways that only someone working with that data would know about, doesn't it make sense to, well, ask one of the industry people who works with that data?

I wonder how many in industry would take the time if asked.

Thursday, 4 February 2016

TPPA reversals

I think it's fair to say that Brian Easton sits to the left of the NZ economist punditsphere, and that Mike Reddell sits to the right of the same. I'm not using punditsphere in any derogatory sense here: they're both serious economists worth taking seriously; they also happen to put their views out there.

In the past couple days, they've both put out their views on the TPPA. Reddell winds up arguing generally against it, though without saying it shouldn't be signed, and Easton in favour, though not that enthusiastically. Both make nuanced arguments. Easton talks about the flow-on consequences of rejecting the deal at this point. Reddell talks about how the layers of bureaucracy to which we may well be signing up will do nothing to improve New Zealand's declining productivity, though he falls short of saying NZ should reject the thing from where we're at. He notes by email that he'd agree with Easton: from where we are, it should be signed. But he's not all that enthusiastic.

I'll remain a fence-sitter as it would take just too much work to come to a strong view on it. My confidence interval on whether the thing's worth signing spans low/mid positive and low negative figures, and it wouldn't be easy to tighten that up. If Congress decided not to pass it and the other partners could then clear out the worse parts on copyright, it wouldn't bother me that much - though the deal on copyright is far better than I'd thought it could have been.

Meanwhile, Stuff polls six celebrities for their views on the thing. Unsurprisingly, Lucy Lawless doesn't like it. News!

Wednesday, 3 February 2016

Copyright chats

My Tuesday night chat with Bryan Crump at RadioNZ's Nights covered copyright and the digital world. The content will be familiar to readers here, and to those who'd taken my Econ 224 course at Canterbury.

My notes for the chat are up at The Initiative's Sandpit blog. An excerpt:
Consider further what infinite copyright would look like. There was a great sci fi short story in the 80s called Melancholy Elephants. In that copyright dystopia, all works have to be run through a plagiarism engine to make sure nobody is copying any prior ideas. And they’re considering making the duration of copyright infinite.
In the Melancholy Elephants story, a senator’s backing a bill that would extend copyright to being infinitely lived. Our protagonist warns him that this would mean the end of the creative world: the regime checking all works against anything that had been created within the copyright period had already killed new creation, because everything builds on everything and just about nothing is entirely novel. Extending it to infinity would mean that nothing could be new and, worse, nobody rediscovering things anew would have that joy: they’d quickly be told that somebody else had had the idea 40 years ago and that maybe they’d heard a snippet of the tune when they were a kid. Never forgetting would mean never feeling the thrill of (re)discovery.
Now that short story is fiction from the early 80s. But the lawsuit by Larrikin Music from a few years ago was not. There, Larrikin had bought the rights to an old Aussie folk tune. Men at Work payed homage to that piece of Australiana in Land Down Under in an 11-note flute sequence. And they got sued, and lost. And, subsequently, the suicide of one of the band members was attributed in part to his dismay at having been thought a plagiarist.
We also covered copyright provisions in the TPPA.

Healthy subsidies

What happens when you give shoppers a discount on healthy foods? Not very much, according to a new field study by Cawley, Hanks, Just and Wansink.

Their mechanism was neat. Shoppers' purchases at a supermarket were tracked via loyalty card; participating households also received a debit card. Depending on the treatment group, they either got a 10% rebate on all food purchases or a 10% wedge between healthy and unhealthy foods that was either framed as a tax on unhealthy foods, a subsidy for healthy foods, or as both: in all of those treatments, the real effect was a 15% discount on healthy foods and a 5% discount on unhealthy foods. Groups were evaluated relative to their baseline readings before treatment began, during which they all received a 10% rebate on all purchases; they then can run difference-in-difference to get treatment effects.

What did they find? No significant effect on actual purchases, but a lot of perceived effects.
In the survey conducted after the treatment concluded, subjects were also asked whether or not participating in the study influenced their shopping. The unconditional means by group are reported in Table 12. Those in the treatment groups (all pooled) expressed greater agreement with the statements that they were buying more starred (nutritious) foods, more healthier foods, and a higher percentage of healthier foods, but the difference between the treatment and control groups is not statistically significant in any of those cases.
There are significant differences in the mean response to these questions by frame. Specifically, those in the tax/subsidy frame tend to express greater agreement that the study led them to buy more nutritious foods, buy healthier foods, and buy a higher percentage of healthier foods, relative to those in the subsidy frame. Notably, we did not see such a difference in our data in the actual expenditures and quantities purchased.  
Some of the lack of effect is due to low power in a sample of about 200.

But they did run a neat permutation analysis afterwards checking to see whether the effects they did find across the groups made sense: they re-ran everything over a thousand iterations in which each participant was randomly assigned to having been in a different treatment than the participant actually was in. And they there found that 70% or more of the random-draw assortment showed larger effects than the ones found in the main estimates that looked at the effects of the group the person was actually in.

What do the point estimates say in a low(ish) power study? Treatment groups spent $1.11 more on nutritious foods and $1.55 less on less nutritious foods per week. If you look solely at lower income households, they spent much more on both nutritious ($7.03 per week) and less nutritious ($7.11) foods: the income effect of the subsidy mattered. Households in the treatment group increased the share of expenditures on nutritious foods by about a percent. That effect was smaller among poorer households because they spent a lot more on both categories.

The biggest effect they find is when they split things out by treatment frame and by income category: low-income households told that the debit card provided them a subsidy for nutritious food substantially increased purchases of less nutritious food ($21.23 per week), with no statistically significant (but still a large point estimate) increase in purchases of nutritious food of $11.58 per week. We should be careful on this one though as low powered tests will only find large effects to be significant.

They conclude:
Taxes on energy-dense foods are arguably the most commonly-advocated anti-obesity policy. The results of this paper have several implications for such policies to promote more nutritious diets. First, taxes may need to be large to change behavior. In the U.S., taxes on soda pop and snacks average one to four percent (Chriqui et al., 2014), but we find no significant impact on expenditures or purchases from a 10 percent relative price change. Second, price changes may have different impacts by income; we find that subsidies for nutritious food may lead low-income households to buy more of all food, including more of the less nutritious food that the policy is attempting to discourage. 
...while giving the usual caveats about drawing inferences from studies that might not have sufficient power.

Tuesday, 2 February 2016

Co-morbidity and the irrationality stick

It irritates me when economists reach quickly for the irrationality stick when they see a behaviour that they wouldn't themselves choose and that they don't like.

Here's Frances Woolley's review of Phishing for Phools. She there takes on their argument that alcohol is more harmful than tobacco:
So it's not that hard to show alcohol is more harmful than tobacco. The trick is to focus on total harm, as opposed to harm per user, and on measures of morbidity, rather than mortality.
The belch in Akerlof and Shiller's argument is that they mostly ignore this evidence on disability, focussing instead on a radically different type of harm: "loss of affect" or "loss of capacity for intimacy". Alcohol, they argue, causes "subjective, hard-to-observe changes in personality", and leads to divorce and the break up of relationships.  The major piece of evidence backing up this position is the Harvard Grant Study, which tracked Harvard students from the classes of 1939 to 1944, and found that those abused alcohol at some point in their lives (23%) died young. Moreover, "alcohol wrecked their ability to relate to others."
Making any kind of statement about the psychological impacts of alcohol is hard, because so many people use alcohol to self-medicate. As Eric Crampton has argued here, alcohol abuse can be symptom of depression, anxiety, or other personality disorders, rather than a cause. Akerlof and Shiller argue that the alcoholics in the Harvard Grant study were, at the start of the study, "no different from their more sober peers" in terms of their personalities and family backgrounds. Instead, "alcoholism had changed their personalities." This reading of the evidence seems to seriously underrate the ability of present or future alcoholics, and their families, to lie and dissemble, and seriously overrate the assessment abilities of late-1930s social scientists. I can believe that alcohol use impacts people's personalities. But I find it hard to believe that, absent alcohol use, alcohol abusers would be just like anyone else.
I'd point to this post on alcohol and co-morbidity.

Co-morbidity is complicated and hard. Just comparing the life satisfaction survey figures, or other outcomes, between users and non-users is hardly sufficient: where some substance use is part of self-medication for prior illnesses, attributing the total difference between users and non-users to use is simply wrong. Could be that outcomes are better than you might have expected, if the self-medication works; could be that outcomes are worse. Either way you need to be looking at the marginal increment against the proper baseline rather than just comparing outcomes across users and non-users.

You might think this is just a problem for crappy cost-of-illness studies that nobody should ever take seriously anyway. But look at the psychiatric hospitals that went and banned smoking. Turns out now that smoking behaviours among schizophrenics is very plausibly self-medication. See also here.

People consume psychoactive substances for reasons. Assuming away those reasons and concluding people are irrational or phools...

Monday, 1 February 2016

Costing policy

My Friday column at The NBR noted some of the potential difficulties in the Greens' proposed policy costing regime.

  • Housing it in Treasury is a bad idea as compared to in an independent office of Parliament, so it would not have to conduct a substantial body of work that it would need to keep secret from the Minister overseeing the agency;
  • A rule or norm that every policy be costed means that it will be more difficult to cost policies than you might expect for currently costed policies. Why? Right now, parties can refrain from costing silly policies. In the alternative, they'll have reason to delay and obfuscate, making it harder for agencies to undertake a costing;
  • New spending at each election can be big, but even bigger is the body of existing policy that is rarely revisited.
And so I concluded:
But the bulk of the government’s spending – the big line items that do not change much from budget to budget – receive rather less attention.
Take the government’s student loan programme. Labour’s zero-percent student loan policy drew attention during the 2005 election campaign and again in 2008. But we do not often hear discussion of the $5 billion gap between the nominal value of the loan and their carrying value.
The vast majority of government spending simply carries on, from year to year, without much notice.
Perhaps an agency established to cost new political party proposals could, between elections, spend its time running rolling reviews of existing spending programmes, ensuring they continue to deliver value for money.
After I filed the column on Wednesday, I found that Mike Reddell had some similar things to say. He also helpfully points to international examples. He also suggests it could be at least as good to simply provide funding to political parties that they then be able to commission independent evaluations. Here's Reddell:
On balance, I still think there is a role for something like a (macro oriented) fiscal council in New Zealand, perhaps subsumed within the sort of macroeconomic or monetary and economic council I suggested here (but perhaps that just reflects my macro background). And there is probably a role for better-resourcing select committees. But when it comes to political party proposals, if (and I don’t think the case is open and shut by any means) we are going to spend more public money on the process, I would probably prefer to provide a higher level of funding to parliamentary parties, to enable them to commission any independent evaluations or expertise they found useful, and then have the parties fight it out in the court of public opinion. The big choices societies face mostly aren’t technocratic in nature, and I’m not sure that the differences between whether individual proposals are properly costed or not is that important in the scheme of things (and perhaps less so than previously under MMP, where all promises are provisional, given that absolute parliamentary majorities are very rare). If there are serious doubts about the costings, let the politicians (and the experts each can marshall) contest the matter.

Friday, 29 January 2016

What is seen and what is unseen: Kitchen costs edition

Christchurch planners failed, post-quake, by not removing the restriction on secondary units in houses. Adding a few walls into a relatively undamaged house with spare rooms and turning one of those rooms into a kitchen would have been the simplest way of adding new dwellings post-quake, when insurance was hard to come by and new builds during aftershocks were ... difficult.

But rules against secondary suites have costs even outside of emergencies. Those costs are often unseen. If your family situation requires a home with secondary units in it, and those don't exist, the harms you experience are unseen by the planners.

Gary Sturgess's not knowing the house he purchased was not properly consented made visible some of these unseen costs. He bought a house with secondary suites built into it because it was important for his family to have them. Then Council told him those units were not consented and he would have to tear them out.
Palmerston North resident Gary Sturgess cannot believe how much red tape there is over a kitchen sink.
The particular sink is part of downstairs rooms at his Ruahine St home where 21-year-old granddaughter Georgia Garrett, who has Down syndrome, is learning independent living skills while under the same roof as mother Tania, aunt Jacqui and granddad Gary.
The family bought the house 2½ years ago, having chosen to live together after a difficult year when Sturgess' wife Valerie had died. 
The home had everything to enable three adults and a teenager to live together for support, while each had private space, a bathroom and somewhere to prepare a cup of tea or basic meal.
But a few months after purchase, a letter from the city council told them the two downstairs units did not have planning consent.
Sturgess was taken by surprise, and appalled by the initial estimates of the cost of putting things right.
Council head of planning services Simon Mori said the presence of a kitchen sink was used as the yardstick to decide whether a unit or sleepout was self-contained.
"The building would then have everything it needs for a person to live there."
And if kitchen sinks were installed legally, it required a building consent, which triggered a review of whether the building work also met planning rules.
The council is in the process of reviewing residential zone rules to give people more choices about building homes and make it easier to have just one dependent living unit on their property, but the changes were unlikely to go far enough to cover Georgia's sink.
In Christchurch's case, I think the real why was that rich folks in Ilam didn't want neighbours to be able to accommodate students, and Council cared more about rich west-side NIMBYs than about poor people in the east who lived in sheds. In Palmerston, I don't know.

If the house had never been built as it was, the harms of Council's policy would be unseen. Sturgess would never have bought the house and would have been forced into accommodations less well suited to their needs. Council cannot anticipate the needs of different families, much as they might like to believe they can fully anticipate such needs and issue detailed directives so that housing matches what they think needs might be.

But Sturgess bought the house. And unseen costs are now visible. The costs of removing the sink?
And it leaves Georgia with a problem about washing her dishes.
"I make sandwiches for lunch, I make my breakfast – where am I going to put my dishes?"
The choices would be to use her bathroom, or the laundry sink, or the upstairs dishwasher – nothing like the conventional habits of independent living.
These kinds of rules should change.