• On Listening to Music at 42

    When I used to blog heavily circa the early/mid-2000s, it was mostly about music. I would come back from a show at 1AM or whatever and then spend an hour talking about what I just saw. Openers described in detail, audience assessments (often about how Vancouver doesn’t support bands that I liked), and usually how awesome the band that I went to see was. Words like “angular” and “melodic” punctuated what was often a semi-factual first person account that was riddled with typos.

    I would see so many bands each year. At my peak, I averaged about 40 gigs a year. I would drop by the record stores every Tuesday at lunch to pick up new releases (Scratch used to be my fave), scooping up 3 to 5 thanks to the new-found riches from a full time job. It was a great time to be around and paying attention to new music, when exciting indie bands were mainlined to you via this new ecosystem created by the internet, eventually leading to many breaking through to the mainstream.

    This was in an era when the word “indie” still had meaning – as in their recordings were released on an independent record label that may not have the distribution reach and publicity machine to promote and sell records. The internet, with sites like Stereogum, PopMatters, and Pitchfork, plus reams and reams of blogs, narrowed that gap, helping build buzz for bands whose previous hopes for a media bump was the cover of CMJ, Magnet, or Paste, maybe a feature in Spin if they’re really lucky.

    When the likes of the New Pornographers, Broken Social Scene, Arcade Fire, Metric, Stars, Final Fantasy, and Feist broke out in Canada, it was helped in no small part by online fawning, of which I was the tiniest of a part. I would do album reviews, gig reviews, and lament about how bands that I love continually get overlooked, not by the mainstream, which was a given, but by indie tastemakers who always seemed to favour brooding male-fronted bands. Rilo Kiley, Azure Ray, Mirah, all those twee bands – they never got their flowers as far as I was concerned. I especially hyped up local(ish) bands like Maplewood Lane, the Organ, and Immaculate Machine given there weren’t many blogs in Vancouver that talked about music. There was also this one local band that I would bemoan all the time because they opened for touring acts a lot, and I always found them boring. Their identity will be left as homework for the reader.

    I was around at the beginning of the hype cycles. I was there for Interpol’s Vancouver debut at a sold-out club that maybe fit two to three hundred people – I didn’t see most of the set but that’s another story. I discovered the wonderfulness of the National only because they opened for the super-hyped Clap Your Hands Say Yeah. I saw the Killers open for stellastarr* at Richard’s on Richards in like 2004. I saw Death Cab for Cutie many times before Seth Cohen dropped their name on the OC, even gave them mediocre vegetarian food recommendations before their gig at the aforementioned Richard’s. I can go on. You know how we old people are like.

    Suffice it to say, in my 20s, music was central to my life, and I was there before most were. That was largely due to the amount of free time I had in additional to the hardcore fixations folks at that age had. These days? I barely keep up with new music from folks I already liked. I do discover new bands now and then, but that’s more or less by osmosis than the active seeking-out I was doing back in the day.

    Music, Now

    I have a different relationship with music these days. When 90% of my time at work was spent coding, music would be playing constantly because that’s the kind of work I can do while the new Au Revoir Simone was (softly) blasting in my ears. Now, I do much more non-coding work that requires more concentration, which is not conducive to having music passively playing in the background. This results in fewer overall spins, which means new music get deprioritized. Further, as the roster of bands that I like get bigger, there’s just more volume to deal with. The aforementioned New Pornographers? I’ve only listened to their new record maybe twice through no fault of the material. I just don’t have time and there’s more to listen to.

    So while I can no longer call myself a hardcore, I still do my best to support an industry that has brought me so much joy throughout the years. My show-going has dropped dramatically (going out at night while having 2 kids is… not easy), as have my overall spending. The convenience of streaming means that’s how I listen to music these days – though I know I am effectively no longer financially supporting the artists that I like anymore through listening to their music. As a result, I do try to buy as much merch, vinyls, and tickets as possible (even if I’m not sure I’m able to go). Anybody that make music that I like and has a Patreon (like Stars), I’ll subscribe to it. One day, I’ll give Torq or Amy that $1000 for a song commission…

    I would say that even now, music is still a central part of my life – it’s just that as a percentage of time and attention spent, its magnitude has dropped. But unlike many hobbies and interests have fallen by the wayside or have been reduced to a minimal trickle (bye bye comics and video games, care-free travelling, and Whitecaps seasons tickets), I’m still listening to new music and going to shows, which is something. Maybe the fact that it’s still happening for me at 42 means I’ll die anticipating the 100th year revival of Lilith Fair? Yes, I plan to live that long. There’s too much good music for me to listen to for me to go that quickly.


  • Why I’m Doing This (Again)

    I still like the positive aspects of social media. Making loose connections with like-minded folks that share my niche interests, getting pithy nuggets off that make me (and a few others) chuckle, and being exposed to random stuff that enrich my world view: all that is made much easier when I’m part of a follow-graph-based network that pushes content to me, feed-style.

    But we are in the midst of an inflection point for social right now, and like many, I’ve been reflecting on its role in my life. I would say I’ve probably been affected more acutely than most in the upheaval, so much of the thinking and self-examination that others have been doing (e.g. should I still do Twitter now given *gestures wildly*?), I’ve already gotten past.

    For me, it’s really about this: what am I getting from Twitter et al, and what am I *not* getting? If one of these Twitter clones take off, I’ll probably be on there, but I realized that there are things I haven’t been getting from Twitter that these other places also won’t give me: the space to stretch my thinking in written form. Tweet threads are fine for takes of various temperatures in bullet points, but the form is just inherently limiting for free-form exploration of ideas in text. It’s hard to really dig into anything meaty if you’re always having to worry about character count.

    Doing Me For Me

    I used to blog a lot in my 20s. I would mostly talked about music, sports, and other pop culture stuff. But really it was mostly music-related. To give you a idea of when, this was before Google bought Blogger. I had a lot more time on hands then – and a lot more things to say, mostly very specific thoughts about very specific things. As the amount of time I had shrank, social media kind of took on that role for me. I’m able to get my thoughts out, not spend too much time when doing so, and have even more folks read it than the few dozen page views I used to average daily. It seemed like a win-win.

    But while socials is great for transmitting random thoughts to people who may or may not want to read them (thanks, algorithmic injection), I came to realize that that’s not really what I want most when I put my thoughts out there. OK, I want SOME people to be able to consume and give feedback on my brain dumps, but the ability to write them down and publish them in the form that best suits my verbose navel-gazing isn’t in 280 or 500 character chunks – it’s on ye olde web logs, under my own domain name, and in my own colours and layout, where I can go on as long as I want. I’m not doing this for other people: I’m doing this for ME.

    Blogging (and its recent spin, newsletter-ing) is an inherently a selfish, self-serving, and self-absorbed activity. Some do it to get exposure, some do it to be part of a community, and some do it to make money, directly or indirectly. For me, and I suspect many others, it’s just a medium to express ideas and talk about things in detail that no one in their daily lives have the time or inclination to listen to pay attention to at the same level of detail. It’s a place for me to talk to myself that also happens to be public.

    My primary objective for blogging is simply to get my ideas down and out of my head. My secondary objective is hoping that others, however few, find some value in those ideas. If that’s not the case, well, at least my primary objective is achieved. This is why I debated long and hard about what platform I should use, as some may lead to easier discovery and a better chance of gaining a bigger audience. But at the end of the day, I just chose something (Porkbun-hosted WordPress) that makes it easy for me to get something up relatively fast.

    What You’ll Get

    This blog is, uh, text-forward. It will touch on many subjects that I’m interested in. You can navigate it using the tag cloud and search functionality that you’ll find in various places on the site. You can even subscribe to it via RSS feeds. But I’ve time-boxed the amount of effort I’ll spend making it reader-friendly, not because I don’t want to make the site as easy for you all to access as possible, but… actually yeah that’s exactly why. At the end of the day, this is an extension of me: verbose, polymath-y, and a bit difficult. Take it as it is. Or don’t.

    Back to that selfish thing: I’m here for me. I’d like you to come along too, but I’m mostly here for me. Just like my tweets, I expect to represent my view points here as authentically as possible, but I only claim as much authority as you choose to give me. If you think what I talk about is worth the read, please keep reading! If you think I’m full of shit, I probably don’t care? I am going to be as factual as I can be, but if you feel that I’m talking out of my ass, maybe you just shouldn’t read this. The only thing I’ll promise is that I’m going to be me, which is either an endorsement or indictment, depending on how you look at it.

    But who really know what I’ll end up talking most about. Right now, I feel like work-related topics like Android Performance and data are going to take up a lot of space. I will likely talk about Watford a lot too, as well as the intersection of numbers and sports: money, analytics, salary caps, maximizing value, etc.. I know I’ll get into some culture/race related shit because that just occupies a lot of my brain cycles. Nothing will be off the table as long as I feel like it’s appropriate for me to talk about publicly.

    So yeah, we’ll see where this goes. If nothing else, I’ve got a cool domain name.


  • On Moneyball and the Real Lessons Learned

    The mystique around the 2002 Oakland A’s was built on the foundation laid by Michael Lewis’ excellent 2003 book Moneyball. In it, A’s GM Billy Beane, handcuffed by a cheapskate owner, had to use non-traditional techniques to identify and acquire players that other organizations undervalued. Namely, he and his team used data over traditional scouting to find good players who were not expensive because they had skills that were not valued. The movie of the same name took the hype to the next level, and soon the word “Moneyball” on everyone’s lips, beyond sports, beyond America, where it was used to describe anything from using data in decision-making to exploiting market inefficiencies to over-perform your budget.

    As the term took off outside baseball, it kind of went out vogue within it, eventually displaced by the term analytics as a catch-all for using data in scouting, player development, and in-game decision-making. 20 years in, using data is no longer novel within baseball – it’s de rigueur. The same goes for other sports, where it’s not about whether or not data is used, but just how much. And while the likes of Bill James were doing statistical analysis long before Beane flopped as a Mets draft pick, it’s hard to argue that he (along with Lewis) brought it to a mainstream audience.

    But here’s the thing: while Beane was notable for his use of On-Base Percentage as an undervalued metric for evaluating players, among other data-based revelations, and making Scott Hatteberg more famous than he had any right to be, his real secret weapon was wage suppression. All of the team’s stars were young players whose earnings were artificially suppressed because they were under “team control” – meaning they can’t negotiate higher wages or go to another team because their rights were held by the A’s.

    The most egregious cases were the teams’ 3 young star pitchers, Mark Mulder, Tim Hudson, and Barry Zito, who made less than $2M combined that season, less than 5% of what was already a paltry payroll. While the A’s were able to improve on the margins cheaply because they valued walks as much as hits, the biggest reason for their success was the fact that their 5 best players – sluggers Eric Chavez and Miguel Tejada were the other two – made less than $8M combined that year. Without that level of extra value created by an unfair system, there was no way those A’s would’ve won 103 games that year with that payroll.

    So the moral of the story is that just because something was seen to be groundbreaking, so much so that it changed multiple industries indelibly, it doesn’t mean the story behind the story is exactly like it was told.

    Never Stop Never Stopping

    When I first read Moneyball all those yeas ago, my first reaction was to find novel market inefficiencies everywhere so that I can exploit them in order to over-perform at whatever I’m doing. But it didn’t take long for me to realize that this was just not it. Even the idea of using data to find nuggets of insight that only you would know, that’s never going to last. If what you do is profitable, important, or otherwise interesting to other people, there will bound to be other really smart people out there trying to do exactly what you’re doing. It’s only a matter of time until everyone else catches up.

    Special sauces might give you a head start in whatever field you’re in. It did for Beane – until the rest of the league caught up. OBP became properly valued and he and the A’s had to move on to a new thing. And when the rest of the league spun up analytics departments too, the A’s actually fell behind because they lack the resources to fully exploit the power of data. When you deal in the realm of knowledge, trade secrets can only take you so far.

    So the lesson I landed on after properly digesting Moneyball is this: in order to create sustained success, you’ve got to differentiate yourself and never stop getting better. The only sustained competitive advantage in an information economy is non-stop innovation. Lead the field and let everyone else follow.

    And on top of that, you have to take full advantage of the environment around you. I don’t want to use the word “exploit” because there is a moral line that I wouldn’t cross, but within reason, you have to leverage what’s at hand. For Beane, it’s about fully leaning into the collectively bargained agreement that MLB and the players’ union signed that allows for below-market salaries to be imposed on young studs – and staying away from inefficient contracts for free agents whose best are behind them. Only by combining those two strategies can you maximize your outcome.

    Epilogue

    For Billy Beane and the Oakland A’s, despite Moneyballing their way to some amount of regular season success, they never could quite make it over the line and win the World Series. There’s also a lesson there: just because you’ve maximized your potential, it doesn’t mean you’ll always achieve the kind of success you want. In addition to luck, sometimes your best just isn’t good enough, especially when you come up against folks that are also doing Moneyball things but have access to a lot more resources. Think the LA Dodgers in baseball, and Manchester City in football. But you have to be OK with that. You do the best you can and let the chips fall where they may. The only thing you can control is the process, and to judge yourself only by the outcome (which you can’t control) is shortsighted.


  • Rethinking the App Startup Metric

    The granddaddy of Android performance metrics is probably application cold start. Setting aside the complications of how to precisely measure it (PY breaks it down really well in this post), lets think about what it represents. Beginning with a user-triggered action, the app process is created by the OS by forking the zygote, leading to the creation of the Application object, after which the Activity lifecycle kicks in (usually), finishing with the UI being rendered, ready to be interacted with.

    This is usually the first mobile performance metric app devs look at when they want to understand how well their app is performing in production. Google treats it as one of the golden perf metrics, important enough to be one of the few that are collected automatically and displayed on the Google Play Console. You’ll even find a litany of blog posts and conference talks about how to improve it. I myself have even spoken about this at droidcon SF on how improving it at Twitter helped us grow DAU tremendously.

    But do users actually care about this?

    Focus On App Launch

    I mean of course they care about how long they have to wait for an app to become usable when they tap the app icon. But what they care about is the app launch time as they perceive it, not what is happening under the hood.

    The only time users care about cold start duration is when they are in the middle of one, and only in the context of it delaying the app launch they’ve started. And you know what is faster than a cold start? A warm or hot one. Google explains the difference between the types here, but the point is that the fastest cold start is to not have a cold start to begin with. So if you want to improve a user’s experience, don’t only focus on how to make cold starts fast – also think about how you can minimize them.

    Ultimately, what users care about is having the shortest app launch time possible. Cold, warm, or hot, they just want whatever’s the fastest. If they had a choice, they’d probably want the device to read their mind, launch the app, and have it be ready when they look down at their phone. If they don’t mind having their minds read, that is. While we can’t quite do that, we can minimize the time it takes for an app to launch – and that goes beyond improving cold start duration.

    One Metric or Many?

    So you might be asking yourself: Is this guy advocating that we collect all types of app starts under one app launch metric, irrespective of, um, temperature? Oh HELL no. A million times no. Don’t you DARE put that on me!

    Munging different operations together into one metric means we can’t understand what actually happens if that number changes. Which workflows got faster or slower? A composite number like that is much less predictive and actionable than its constituent parts, making it more of a vanity metric than if we were to track them separately. So in order for us to measure precise changes to each workflow, we cannot collect one “app launch time” metric that includes all three. These must be different metrics.

    That’s not to say we can’t somehow remix the three app startup times into a singular metric somehow. In fact, if you do it right, it can be very powerful. Like the idea of OPS and OPS+ in baseball, there could be further insights gleaned if all the app startups are combined in the right way. But the constituent parts have to be measured separately so that you can determine the underlying changes and take appropriate action if warrented.

    So what’s a good way of doing this kind of metrics remixing?

    A Dead-End (For Me)

    One possible way is to simply smush the datasets for all three types of app startups together, basically the munging I had so objected to earlier. But this time, we will also have each tracked separately. The biggest advantage of this is that it’s easy. However, its usefulness may be limited given the slower startup times dominate the formula, and depending on the distribution of your particular app, the trend for this combined metric may not be far off from just what cold start is.

    You can also do a weighted average to dampen the effects of the dominating component – but how will you choose the weighting factors? I suppose you can play around with different ones, and see if you can find correlations with other metrics, then do experiments to validate the relationships. But this process depends heavily on the underlying user base and distributions of data that it produces, so not only will the results not be generalizable, they may change without you even knowing.

    Perhaps better folks than me can use this methodology to derive sustainable value from this type of Voltron-ing of the various app startup times, but it may be beyond my capabilities at this point. (If you’re able to do this, please blog about it or just ping me because I’m dying to know!)

    A Possible Way Forward?

    The one combination of startup metrics that I’ve been noodling on, one that I’ve yet to prove is useful with real data or convinced myself will be too difficult to make work, is creating a ratio of cold starts over total app launches and tracking that as a metric to represent how often a full cold start is required when a user launches an app. We can call it… the cold start rate? The higher this number, the worse it is for the user, so the idea is that it should be kept low (or be reduced if an improvement is sought), and any material increase should be treated as a regression much like how folks treat cold startup time regressions right now.

    Will cold start rate be predictive? It’s hard to say without data. Will it be actionable? Maybe – if you can see why the ratio has changed, and have other metrics that would give you clues as to why that maybe the case.

    Perhaps you have a memory leak and lmkd is more aggressive with killing your app when it’s in the background because of the process’s higher oom_score_adj value. Seeing a slight increase in your Out-Of-Memory-Exception metric combined with a higher rate of cold starts might lead you to fire up LeakCanary to find the leak to address the problem. In that hypothetical scenario, the increase in cold start rate gave you proof that users are impacted by the leak, and that fixing it also fixes the regression. Simply tracking how long cold starts take will not give you that insight.

    Anyway, contrived example aside, the point is not whether this cold start rate metric turns out to be something generally useful. The issue to highlight is that the way many of us monitor app launch times is kind of incomplete, focusing on the absolute value of the slowest kind of app launch, rather than optimizing for the general case. We absolutely can and should do better.


  • On Sustainability

    Watford FC has enjoyed several years of Premier League success followed by several years of what can generously be described as mediocrity. This last period of non-success (promotion notwithstanding) have not only been taxing on fans, but on the bank account of the club too.

    Every year we were not in the Premier League, we lose out on roughly £60-70M of revenue from broadcasting, match day, and commercial sources. And that’s after parachute payments, and not counting the discount we take in player trading because we have less leverage to demand higher fees.

    And despite record turnover, we weren’t breaking even during our run in the Premier League between 2015 and 2020. In fact, Gino had to lend the club an additional £35M in the 2017/18 season so we can increase our investments in the squad, and even more after the first relegation. But we made it work thanks in no small part to some shrewd player trading.

    Financially, we were relatively sustainable with our strategy of buying young players from undervalued markets and selling them for profit (with or without them playing for us). But that only worked because we were in the Premier League, to attract top players, fund the spending, and foot the bill for transfer deficits and other costs associated with having source club funding externally.

    It’s that last point we haven’t not talked about enough as a fanbase, the reason we cannot spend very much at all in this transfer window: we have bills to pay, and no other means of paying them other than reducing the total amount of funding to the club. The money we bring in from selling the likes of Joao Pedro need to go to loan payments and past-transfer fee instalments. And we can only afford to do that while covering wages and expenses because of parachute payments.

    Club Funding and Relegation

    I can probably write a whole post or two on how football clubs are funded. I’m not talk about paying wages and operational expenses, but rather investments in infrastructure and players. There’s probably a more specific term to describe this – it’s a combination of working capital, share equity, outstanding loans, and net transfer payables, all going to fund operational assets like player registrations. It’s money tied up with the club so that it can operate.

    Premier League clubs can sustain a large amount of club funding even if their owners are wealthy benefactors with hundreds of millions invested in the club via share equity and shareholder loans. Higher revenues allow them to spend more money on player transfers, sustain a higher level of loan interest payments (and borrow more), and be given more leniency in deferring transfer fee payments – so more money can be tied up in operation assets. With relegation, the level of funding a club can sustain shrinks precipitously, and how a club manages that reduction says a lot about how financially responsible their owners are.

    Clubs that want to “bounce straight up” try to keep funding at an unsustainable level. They largely keep their squad intact, or reinvest the money they recoup on player sales in new players. They may take out riskier loans with higher interest rates to get the cash in to make up for the shortfall of revenue. They’re basically trying to hold it together so they can resume their regular levels of spending if they get promoted again.

    This is not a terrible strategy to take for one season if you’ve had a long run in the Premier League and your finances are in decent shape. This is the path that Gino chose after the relegation in 2020, at least a cautious version, and it succeeded! Somewhat.

    The problem with stretching yourself thin like that is it reduces your ability to improve the squad responsibly even if you achieve promotion. You’ve effectively already spent some of your future revenues to fund the promotion, so you have less to spend to stay up. If your existing squad is already pretty good, and you don’t need to buy a tonne of players in order to be competitive, then congratulations: you have a fighting chance of Premier League survival.

    The squad Watford was promoted with in 2021 was not Premier League ready. The money we were able to spend did not change things materially, and that’s with getting a bargain in Dennis and essentially having Cucho come in for free. That midfield and defense were not close to being good enough after years of poor recruitment, and we simply did not have the money to make it Premier League quality.

    With the sustained funding in 2020/21, Gino basically gambled and won. But the prize we got was a miserable campaign in the Prem that ended with an expected relegation. Insert Thanos “But at what cost?” meme.

    Relegation Redux

    With the second relegation, Gino literally could not afford to go down the same path. He kind of sort of tried, but with a squad that had to be weakened further, one that was already worse than the squad that was relegated in 2020, it was always going to be tough. The quick pivot away from Rob Edwards did not help, but that was not the root cause. The players simply weren’t good enough, and we couldn’t spend the money to make up the difference. The fact that the team underperformed even that lower bar made for a horrendous season for us fans.

    Facing a second year in the Championship, I think the overall strategy has changed, even if Gino is willing to admit it or not. Neither he nor Scott have been talking up promotion this year, and this is because they know it’s unrealistic. We have no money to improve the squad, so the best thing to do is consolidate at this level of funding and build back up with younger players who will hopefully be ready for the Premier League if we ever get back there.

    The money from Brighton for JP will go to paying off Macquarie, reducing our club funding even further. It will also bring us to some definition of being “debt-free” in 12 months, as Gino said. We will still owe money to vendors as future payables, to other clubs for past transfer fees, and also to Gino, who will once again be the primary source of funding for the club. But there will be no money owned to banks that requires interest payments, which means if we can keep the cashflow even with our player trading and normal Championship-level turnover, the club will remain financially sustainable going forward.

    I don’t want to gloss over the fact that this is really good news for Watford FC fans.

    The one thing that has always worried me and a lot of supporters is the existential threat of the club going out of business because of poor financial management. This is the main reason many don’t clamour loudly for the club to change owners – it’s because we don’t know what’s behind that door. You don’t have to go far in the EFL to find examples of perfectly fine clubs being driven to the ground by owners who overspend and then lost interest. Or just bad eggs looking to cash in on some juicy real estate. Having Watford Football Club to support in any form is the most important thing, bar none, and if that means having to adjust my expectations, so be it.

    This Season

    Now, about that. What do I expect in 2023/24? If I’m being honest: very little.

    Very little in terms of transfer spending. Very little in terms of hope for promotion. Very little in term of chill from the fans because Watford are reverting back to what I’m mostly used to us being for the better part of the last 3 decades: mid-table in the second tier, with a bigger chance to be in a relegation fight than a promotion push.

    All this talk of South Korean internationals and or up-and-coming hotshots in the EFL? It’s all just fantasy to me because I don’t think we can outbid anyone. We’ll get players no bigger clubs want, players that are either unproven, have obvious flaws, or come really cheap. Manga and Costa will be shopping in the bargain bins, which will test our patience as well as theirs, but we have to live with that because we don’t have the financial means to do better. We might get one marquee signing, but I wouldn’t hold my breath for a blockbuster.

    So strap in, Watford fans. This year is going to be like no other in the Pozzo era. At best, hope for consolidation and improvement at the margins, building towards something better with a younger squad we can get behind, a distinct playing style under one head coach all year, and no financial trap doors that make us worry needlessly. If we get all that but still finish 13th, I will ecstatic.