00:00:08 Surma: I think we've set a new record in terms of time passing between podcast episodes.
[00:00:20] Jake: Yes, between the episodes, certainly. We've still got that gap from when we were born to the first episode. Actually, I don't know when you would measure that from once the--
00:00:29 Surma: I do remember that the first thing the doctor said to me is, "Your time starts now."
00:00:33 Jake: Yes. It's been a long time. It's been since November. I also just want to say a lot has happened.
00:00:40 Jake: Yes. One of the things that's happened is that you left me for another company.
00:00:47 Surmer: I have.
00:00:48 Jake: Yes.
00:00:49 Surmer: I don't know how much you want to talk about it. It feels very self-indulgent, but, yes, I left the Googles and went to Shopify.
00:00:57 Jake: Yes, I am really interested. We have met up and chatted outside of a podcast.
00:01:05 Surma: No, Jake, we only talk through the podcast. [chuckles]
00:01:09 Jake: Oh, does the-- oh, that-- On the note, I've been doing the two or three episodes with other people.
00:01:18 Surma: Yes.
[00:01:19] Jake. Yes, I did some episodes with Ada and done some episodes with Cassie as well. Cassie Evans. People are furious about the-- No, two weird guys are furious about the Cassie episode because I dared talk to Cassie about things which weren't animation. They have like they're-- they say, "This is a waste of Cassie's time and my time and everyone's time. How dare this happen?" It's really weird.
00:01:51 Surma: Yes, "How dare you talk to someone who does web development about things that are related to web development?"
00:01:56 Jake: I know. It's a kind of weird gatekeeping, isn't it? I'm pretty sure they think they're doing the white-- they're protecting--
00:02:04 Surma: The White Knight. There's some good white-knighting going on-
00:02:05 Jake: It's white-knighting.
00:02:06 Surma: -deciding that you are wasting Cassie's time which, as far as I know, is still Cassie's decision.
00:02:11 Jake: I think it is, you know. I think, knowing Cassie, I think she would have told me if she felt like her time was being wasted. [chuckles]
00:02:21 Surma: She would have walked out on you.
00:02:22 Jake: [laughs] Absolutely, absolutely. It's strange, isn't it? Because where were they when you were telling me about dithering. That's not my field of expertise, how dare you waste my time talking about-- I can't think of any. I'm sure you did one on WebGL or something else that wasn't' my field of expertise. I did a whole episode about keyboard events with Ada. It just seems to be really weird. Some sort of crossover of things that has happened where it's like "I think you'll find Cassie only talks about animations", so you're only allowed to do-- Anyway.
00:02:59 Surma: I think you'll find you have to pigeonhole Cassie, and if you don't do that, you're offensive.
00:03:04 Jake: Exactly. I was worried because me and Cassie went to the pub to discuss the episodes and that kind of thing, and we talked about all sorts. I'm so thankful those guys weren't there because they would have been tearing the place apart, smashing glasses and plates [unintelligible 00:03:18] bright red in the face. "What? You're just talking about TV shows, she does animation. She does animation. How--"
00:03:26 Surma: She always talks about animation only.
00:03:29 Jake: I know. Anyway. Sorry, we were talking about you.
[laughter]
Sorry, I was literally wasting your time there by talking about things that you're not expert in. Let's talk about Shopify.
[laughter]
00:03:43 Surma: Yes, it's a company, and I work for them now.
00:03:52 Jake: Are you still there?
00:03:54 Surma: Yes.
[laughter]
00:03:57 Jake: You said "now" and usually when people say "now" that's them leading onto saying something else.
00:04:03 Surma: Oh, all right. I work there now.
00:04:06 Jake: Oh. I heard like, "I work there. Now--"
00:04:09 Surma: Now-
00:04:09 Jake: [laughs]
00:04:09 Surma: -here is all the gossip about Shopify. No, I--
00:04:12 Jake: Now, let me tell you--
00:04:13 Surma: I'm not quite sure what would be of interest. I joined as an engineer working on anything to do with Developer Experience, and that is intentionally vague. Shopify has been growing at an absolutely wild rate over the last couple of years. You know they have this percentage thing; How many people have been newer [unintelligible 00:04:43] than Shopify? When I joined up, 0% were newer at Shopify than me. Only two weeks later, it was 2.5% and now, it's-
00:04:54 Jake: Oh, wow.
00:04:55 Surma: -16. 16% of people at Shopify are newer than me, and I've been there four months now.
j[00:05:01] Jake: That could also be like you joined and then a bunch of people were like, "Right, well we're out." [crosstalk]
00:05:08 Surmer: Sure.
00:05:08 Jake: Also, that's what happened to the stock price, I seem to remember. You joined, and the stock price just absolutely--
00:05:14 Surmer: Coincidence? I think not.
[laughter]
00:05:16 Jake: One of the things I'm actually interested to hear about is the interview process. Not just at Shopify, but general, like job-hunting because that's something I've not done in nine years. It's something that you haven't done for quite a long time as well.
00:05:36 Surmer: It was actually really interesting. We both have talked about interviewing at Google quite a lot, both on the podcast but also between us because the process at Google has its edges, its sharp edges that we are not always happy with. We were always talking about how much liberty can we as individual interviewers take to make the process more meaningful or better without manipulating the system or giving an unfair advantage to anyone.
I did interview-- I didn't get this job through the back door or anything, although I did get interested in Shopify because our former director Dion Almaer, is now at Shopify and I kept in touch with him. When he talked to me a bit about what his job entailed, or what his team works on, and what kind of things they need, that's how I got even interested because eCommerce is not something I know a lot about or used to be very passionate about, to be honest.
We had an eCommerce team at Google as well, a [unintelligible 00:06:41] team even, I guess or at least people working in the vicinity of that. It just never really tickled me in the right way. Actually, the company wasn't really on my radar, but then through Dion effectively, I read up on it and heard from him about how they work and what their goal is.
Basically, I then went through the normal, formal interview process at Shopify. and it is sufficiently different from what Google is doing. The first thing that was noticeably different is they are quick. They move fast. When you say you want an interview, they send you an email-- at least for the couple of processes I have observed and experienced, they tell you a day later to choose from these couple of dates when your interviews could happen. That's a pretty quick turnaround. Similar to Google, it was five interviews, individual interviews, with very different focal points. The first two were like a-- what was it called? It was like pair programming, basically, but in the end, it's a coding interview.
00:07:56 Jake: I was going to say, I'm not very good at pair programming. That's a real weakness of mine.
00:08:00 Surmer: Actually, I think you're right because we used to do it the office quite a lot. It's just you sit with someone. One person is coding and sometimes the other person throws in, you made a mistake or challenges why you made a thing a certain way. Then you have to justify it, or the person who's coding asks a question because they don't know exactly how to continue on this certain path. It's just like one person is coding, but there's two brains involved. It's not like the NCAS, two people using one keyboard or anything like that.
00:08:35 Jake: I just find in pair programming, I either feel like I'm being dragged along by the other person or I'm dragging the other person along. I never feel like either is helpful. It never feels helful to me when I'm being dragged along. I find I get much more out of code reviewing than I do pair programming in terms of me learning a thing.
00:09:04 Surmer: I think, in general, I agree. For me, "Lets start a coding session and work through this issue" is not something I do very often mostly because I move quite fast, and write messy code and clean up later. Many people do not work that way, so that makes it incredibly incompatible because I need to slow down and take the other person with me, but that makes me lose my train of thought. It just goes against my intuition, I suppose. What I do like, and it's something that the two of us have done in the office is that if someone hits a problem, you pull someone over, and you rubber duck. Then you explain the problem, then you tackle the problem together.
00:09:45 Jake: Yes, absolutely.
00:09:45 Surmer: That I think is very helpful.
00:09:47 Jake: You'll scribble it on a whiteboard or whatever. Yes, absolutely. That doesn't count as pair programming to me; that's kind of pair problem-solving.
00:09:58 Surmer: I would file that under "pair programming". That's currently how we're doing it. That's actually really interesting that at Shopify which-- the company is entirely remote, so everybody is working from home. They basically let go of most offices at the start of the pandemic and went to what they call a digital by design principle, that everybody is remote and works from home or from wherever they want and the company accommodates that.
One of the things they have is a program called Topple, which is, it's like Google Meet, but it's optimized for screen sharing and pair programming where you can take over the other person's mouse and keyboard or draw on the other person's screen to be like, "Hey, you made a mistake here." It's really good. I genuinely enjoy it. It gets like how Google Meet is really bad at sharing a screen at a high frame rate and it makes things janky, this tool doesn't. I don't know how, but it's super responsive most of the time.
00:10:53 Jake: I have to say, it's that kind of pair problem solving is something that I have never managed to recreate remotely properly. I did some spec work in February on the history spec. Three of us went to Boston and sat at a room to do this standards work. It felt like we got more done that week than we'd got done like the six months a year beforehand because we had the whiteboard. It was easier for three people to talk at once and manage that without someone having to press the hands-up icon on the chat thing.
00:11:35 Surma: I think that will remain true. FaceTime is still irreplaceable. At Shopify, they do try to do what they call bursts, basically offsite getting-- in-person getting together once a quarter. Now that travel is more and more possible again to have that, but this Topple thing actually makes it sufficiently good that on a regular basis, I jump on with some of the other engineers on the projects I work on and when they have a problem, they get stuck or I don't know how to do a thing, you jump on a Topple call and just help the other person or get help from the other person and it works really, really well.
That was a really interesting experience to see that, how that is just part of the culture. You're like, "Hey, you want to Topple this?" "Yes, let's do it." Then you actually do move quite quickly through a problem if the other person has expertise or you have expertise with the other person. That's really cool.
00:12:30 Jake: What's the kind of-- when you were doing coding interview-style stuff like? Because you didn't just interview it at Shopify, right? Did you go to other places as well?
00:12:38 Surma: Yes. I had a couple other ions in the fire, but in the end, I think only with Shopify, I decided to go through the full process in the end. The first coding exercise was admittedly really simple. It's basically, it was like here, it's a little, little, little app, little game you're supposed to build, go, and you can use whatever you want.
00:13:05 Jake: What's the little game? Was it Minesweeper because that took us more than an interview's worth.
00:13:10 Surma: Exactly. You make trade-offs, right?
00:13:12 Jake: Yes.
00:13:13 Surma: In this case, it was basically you have a grid and you have a-- [crosstalk]
00:13:19 Jake: It's already sounding like Minesweeper mate.
00:13:20 Surma: Yes. It is a grid. You have a player in the grid and you build buttons that can make the player move around the grid. Basically up, down left, right. Then you add support for collision with walls and then obstacles and then in the end you automate to find a path.
00:13:38 Jake: Oh, nice.
00:13:38 Surma: That was the level. It's actually, it's really nice. I was like, "You know what? For the sake of moving fast, I'm just going to in a HTML out of this. I basically just wrote the game engine. If you want to call it that, I didn't use React. I didn't use any of the frameworks, even though I guess with React, I would've scored points because Shopify is a whole React shop as of now but I just basically updated [unintelligible 00:14:07] in our HTML, and just assembled a table markup whenever a button was clicked and it worked.
00:14:14 Jake: It's one of those cases where an HTML table is the right thing to use because it's--
00:14:17 Surma: I assume so.
00:14:19 Jake: I think so. It's a two-dimensional-- It's maybe difficult with games, isn't it? Because it's not literally a data table, but it is one of the two-dimensional, accessible structures that we have.
00:14:32 Surma: Yes. I did ask if this needs to be accessible and they said no, which is then fine. I guess if they said yes, I would've had to think a lot harder whether the table was the right decision here or not, but in this case, they apparently didn't care in the context of the interview. That was like an hour-long interview. The second part was, and that I thought was really interesting, and I wonder if that's something that Google could learn from was a so-called life story where the recruiter that had an hour-long chat with me and just wanted to hear my life story and it will just become a dialogue about my path from where I started to where I am today and what drove me.
00:15:18 Jake: What do you think, what are they like scoring on there? Is it just like a general, "Can you communicate?"
00:15:24 Surma: I think it's what drives you as a person but also the way you present your work, how well you're able to present your work. Yes, communication whether you're a team player, whether you're an egocentric asshole. There's many flags that probably could come up in the way you tell your story. It felt honestly really weird because, again, super self-indulgent just like, "Oh, and then I did this and then I did that and it was--" On the one, it was really nice actually to revisit all the things I had done because many things I had forgotten about. Then through the process of telling the story, I remembered. It felt also a bit ecocentric.
Also, the first two interviews and then the other three were another coding challenge, which was this time a bit harder. I'm trying to remember what the question was. I don't quite remember.
The fourth one was a systems design question. This was not coding, but more of like, "Here is a product we want to build or a little addition to a product. How would you design this?" This was actually a bit more of a full-stack question, not necessarily, but how do you build the backend but more about like the API design and the requirements for the implementation of an API. Like which of these APIs endpoints need to be real-time? Which of these things could be just JSON? What would be a WebSocket? How would you consume this? And so on and so forth. That was arguably the interview that went worst for me. Mostly because I haven't done that crap in a long, long time. In our job, we have built static sites. We specialized for the last couple of years in static sites because they were so performant and haven't really built much with dynamic back ends.
00:17:22 Jake: Yes. That is one of the things where I feel, I don't know, a little bit left out in the industry because yes, we build a lot of stuff static. Things like Squoosh, Prox, it's all static builds. I find mostly when I want to have a server component, it's because I want to use WebSockets in some way and that just doesn't exist in that model as far as I've seen. Once you're on the new generation of server framework, things like Edge, that kind of thing, they're not set up to do WebSocket stuff. I always just fall back to either setting up my own cloud instance or a thing or use glitch.com if it's just like a demo thing because yes, websites don't seem to really fit into any of these edge-driven models.
00:18:25 Surma: Well, now there's Deno Deploy.
00:18:28 Jake: Deno Deploy. Does that do WebSocket though? I haven't--
00:18:32 Surma: It does.
00:18:33 Jake: Or maybe it does because I've not looked at that. Oh, I tell you what the other thing I tend to need is storage. I know Deno's been looking at that, essentially using something like index dbo local storage as their storage system, but I don't think it's there yet. Is it?
00:18:54 Surma: No. I think for now it's something we can-- Let's do tangents. We like tangents, right?
00:19:01 Jake: Oh, we do.
00:19:02 Surma: I took on an advisory role at Deno where really serendipitous story. I don't know.
00:19:12 Jake: Serendipitous, let's go with that. How would I know? I said specificity in a video at the time. I didn't even realize I was saying it wrong because its specificity.
00:19:23 Surma: It is specificity. [chuckles] Any CSSDA at Google needs to say that 10 times fast in the interview.
00:19:31 Jake: Absolutely.
00:19:31 Surma: I've heard from you now.
00:19:32 Jake: That's the whole interview. You've got to do it in one.
00:19:35 Surma: [laughs][crosstalk] If you mess up once, you're gone, you're out.
00:19:37 Jake: Hiring's really hard.
00:19:42 Surma: Randall reached out to me basically two days after I signed with Shopify and I was like, "Hey, so we saw your Deno video because we did a Deno video on two or three-- Apparently, they were looking to step up their game around outreach and advocacy. I was like, "Oh, that's nice." He was talking about, "Well, I want to join the company." I was like, "That actually sounds really interesting." I signed a contract. Luckily Shopify allows, like, in the contract, it allows side hustles as long as you're not helping a competitor which at Google wasn't allowed. You would have to get an exception which I've never even tried, but it was nice to have this.
I was like, "Yes, how about we don't do this as I don't join your company, but I would love to help you as an advisor and maybe we just have regular syncs, meetings with you and the team and I can listen to what you are thinking about and given my two cents," which is basically just me having opinions and I can always have opinions. That sounds fun.
00:20:51 Jake: Opinions for money.
00:20:52 Surma: Exactly, but in the end, I spent the last seven years at Google.
00:20:58 Jake: Oh, that's my job. It's a lot of people's job in the web industry is opinions for money, especially the higher up you get, it's more opinion for more money.
00:21:10 Surma: We spent a lot of time understanding the web platform and developers and how these two things are moving with each other or against each other and what developers need and what they should want but don't actually want and how the ecosystem is evolving, that all was part of DevRel. Over seven years, you do build up a bit of experience, and since Deno is really doubling down on being the web platform, but on the edge, that seems to be a useful skill for them.
I've been working with them for a couple of months now, and it's been really interesting. They are building something what I think will be really cool which is like a really interesting edge runtime where you as a web developer come in and you already know it. There's not much to learn for you. You can just straight-up start building, and I think that's really cool. It turns a web developer into a full stack developer just by implementing the same API to the same semantics.
00:22:18 Jake: We talked about it in the last episode, I believe, because I built the-- in my intro to the world of cause--
00:22:26 Surma: Oh yes, you did-- [crosstalk]
00:22:29 Jake: The server-side component, so that was Deno Deploy and yes, it worked pretty well. If you can do storage on WebSockets, then really that is my other use case done.
00:22:44 Surma: You can always do something like go to the free tier at PlanetScale or Redis or whatever and plug in an actual database that's always possible.
00:22:55 Jake: Yes, as I said, I would probably do it because all the things that I've built like that have been like a tiny little bit of state, but that state needs to be broadcast when it changes, and so that was like big web quiz like what we used to do in-person CDS, that's all that is. WebSockets, tiny little bits that I would like to survive if the server rebooted. I recently built a game which was just a reaction test game, but it had a scoreboard and so there was a scoreboard screen. Again, there it is, a tiny little bit of state WebSocket broadcasting. Job done.
00:23:32 Surma: What was this game for, Jake?
00:23:34 Jake: Oh, yes, that was for Google I/O because Chrome now sponsors the Formula 1 team. I like Formula 1 so I obnoxiously as I could inserted myself into that part of the team.
00:23:48 Surma: And you succeeded.
00:23:50 Jake: I succeeded, and that's the main thing that matters. Google I/O had a McLaren F1 car and next to it was this reaction test game which I built in a week really with the agency doing the design the week before. It was like a two-week turnaround. It was fun because I had all these big ideas. When I got into the meeting, I was saying look, "Here's all the things I think we can build in partnership with McLaren." Then they were like, "Oh, that sounds great. How many of those can we have in time for Google I/O in two and a half weeks." I was like, "Ah, none of those." [chuckles] Let's just sweep all those to one side. I once built a reaction test game with Formula 1 lights, so we could just do that, but here's the thing again, and we did but we did it better.
00:24:44 Surma: When you build it for the first time, even then it got into the hands of real-world Formula 1 drivers, didn't it?
00:24:51 Jake: Yes, it did. The only thing I've ever built that went truly viral and now anything else I ever built will never be as successful as that one hour I spent on that reaction game.
00:25:04 Surma: It's probably with a peak example of the infamous effort-to-success ratio.
00:25:10 Jake: Absolutely, yes.
00:25:12 Surma: Success to effort is probably better because it was incredibly high. We always say this about blog posts, the less effort you put in, the more successful they usually are, or the more popular they get. The things you care about nobody else cares about it at all.
00:25:27 Jake: No, never goes that way. It's not a game I can just put online because here's the thing, there's no way of putting cheat detection into a game like that because the reaction test.
00:25:41 Surma: It runs on the client-side, doesn't it?
00:25:47 Jake: There's a lot of games that run on the client, but the server can verify that yes, those moves were valid. With a game like this, all it's saying is, "Yes, you did it in 200 milliseconds." There's no way of verifying that. Even if you're sending that data over to the server, then whole pings just destroy the data because it's so fast, so yes, it's only ever going to be something that will be run locally at events. I'll probably make a no scoreboard version of it.
00:26:22 Surma: And you'll stand behind the people scrutinizing them if they're cheating and if they are they'll get shot.
00:26:27 Jake: Well, there's been a talk of could we run the game in the F1 Paddock, and I've been making it very clear that, yes, this will need an engineer on-site who gets familiar with the code base in order to make sure that there's no cheating going on, but they haven't taken me up on that yet.
00:26:44 Surma: Who could that engineer be?
00:26:46 Jake: There's only one person familiar with the code base, that's the thing.
[laughter]
00:26:53 Surma: Let's see if that works out.
00:26:55 Jake: Yes, I'll keep trying you never know.
00:27:00 Surma: To finally finish off the Shopify interview story.
00:27:03 Jake: Oh, yes.
00:27:05 Surma: Yes, systems design. I think I did okay, just not super great mostly because I literally missed a couple of opportunities. For example, we were talking about how much we could cache certain API responses. I just missed a very obvious answer that because in my head, caching times are set at compile-time which is wrong. If you write a server, you can calculate them on every request, but because we have been thinking about static builds and deploying to Netlify as a static site so long for so many years, I was just like automatically thinking I need to set caching headers at compile-time.
00:27:49 Jake: On the server invalidation can be dynamic, right?
00:27:52 Surma: Exactly.
00:27:53 Jake: It can know when to flush the cache
00:27:54 Surma: It was like, "Okay, we know that new data comes in on the hour every hour, what do you say with the caching header." The obvious answer is, well, you see how much time is left in the hour and that's your caching header. I just didn't see that answer until she literally pushed my head into it because I was absolutely stuck in static site. That was really interesting just to see how much.
00:28:25 Jake: I will still say that if you're pushing data on the hour every hour, like setting a cash header that is like-- I don't know, an hour minus a bit of safety, that still seems incredibly risky to me. It seems like a decision you'd make in advance and then regret it later because uh-oh, we've now got a bit of data that is saying something accidentally rude or something that is dangerously factually incorrect.
00:28:54 Surma: This goes for API responses.
00:28:57 Jake: I think on the server, the answer is-- hold on where is this caching happening, is this caching happening on the server or is it caching on the client?
00:29:12 Surma: In this case, it was an HTTP caching header what value.
00:29:15 Jake: Oh, I see, but that still leaves you with the thing of like some clients have dangerously outdated data like you thought, "Oh, it's going to be 40 minutes until we're updating this data, but then something happens like I don't know a president dies or something, the queen dies or whatever right one of these things where all of a sudden data that seemed innocuous now seems really insensitive and then you've got clients that are now waiting 45 minutes in order for it to no longer be offensive or something. You know what I mean?
00:29:54 Surma: Yes, I think that that's probably a valid concern to bring up. That is always an issue with these caching headers. I think the answer would probably been, let's not worry about this in this specific use case or something like that.
00:30:06 Jake: Yes. I think like a more valid thing, like if the API call is expensive, then caching on the server seems like a better solution or even caching on the edge because in a emergency situation, you have the calls to go and flush those caches.
00:30:24 Surma: Yes, at the same time, you can always also deploy a new front-end version where you add cache busting to the API call.
00:30:33 Jake: Yes. Well, it depends who's in charge of that, who's in charge of those, if it's like third parties I guess it's harder.
00:30:42 Surma: Yes, no, in this case, it was like we are building a front end and we need to also specify the API contract.
00:30:49 Jake: Fair enough then, so you can have an emergency, like a key that goes along with all of the requests and in an emergency, you changed that key to do that.
00:30:59 Surma: I think even in this use case, something being offensive was fairly unlikely. Anyway, so that was the system's design and it was just an interesting thing for me to realize that I hadn't built like full stack stuff in a while. It's something I'm doing a bit more often. It's nice to get back to it. Like a lot of things have changed and I'm definitely rusty. Then the last interview was a deep dive where they basically wanted me to select a project of mine that I have worked on and know a lot about and just explain it, show it off to an interview and they would ask questions about it and wanted to hear what I did.
I basically just talk an hour about Squoosh and all the interesting things that we had to solve and that we encountered from switching, from React and from web hack with Preact, to Rollup and Preact, to web assembly to our mobile story, our caching. We build a good project there. It has a lot of very interesting facets, but it's genuinely nice to see that it's actually still popular, even though I think it's about time to give it some love again and neither of us really currently have a lot of time to spare.
00:32:14 Jake: Well, because the people working on it were like actively, recently, it was me, you, Jason, and [unintelligible 00:32:21] and now at Google, there is only me left. That's true. I got other stuff to do.
00:32:29 Surma: Well, I genuinely want to still work on it. I still have access to the repository. I asked if I can remain on the repository, but now it's just a question of time and coordination and motivation, I suppose, but it's one of the projects I'm very proud of.
00:32:47 Jake: Oh, yes. It is on my backlog is to go and talk again to the AVIF folks and the JXL folks and say, what's your deal? Because we haven't updated those codex in a while, but updating the codex is one thing, but it's more about like, "Okay, what settings have you've been looking at recently? What's new there? What can we give developers to tweak these things?" That's the bits I'm really interested in. It's always more than just updating the get hash of the code-- [crosstalk] Exactly.
00:33:23 Surma: I wanted to check how AVIF is doing on the web. If there's been significant pickup, I basically switched over to it on my blog still with the fallback, but I think once Safari ships it, I can even remove the fallback. I think JXL still hasn't made it into stable or has it?
00:33:43 Jake: No, it's not been, it's not in any browser yet. The studies they put out, they make big claims about the quality. It's not something I've felt myself, but their study's a scientific minor or not, but then also, I still-- every test they do is based on 1X screens and I just feel like most people on the web are not looking at a 1X screen.
00:34:08 Surma: Well, that was always for me where I feel like JPG Excel was designed for the web, but not exclusively, so I feel like that's always their argument. It's like, well, we also just want to recompress existing images and maintain the quality and it's like, "Okay, fair enough, but the two of us always committed specifically from the web angle.
00:34:27 Jake: I think that there's an open goal there for a Codek to make full use of high density because I do think that the way you would compress the images is different and we already know that you can go for lower quality and get what I'm pretty certain is same perceptual quality because all of the garbage, all of the effects are smaller. They're harder to see but I do think like, is it just a matter of doing the same thing and it works, or is there different settings, is there a different--?
00:35:05 Surma: I think it's really interesting something. I probably, I don't know if that's been looked into and I definitely know that most, all of the metrics, like the computational metrics to measure image difference that we have. Even the one that claimed to be psycho visual, like DSM still take the source image and the compressed image and do math on a pixel by pixel comparison, which on the web isn't necessarily true. The original could be 4X or 5X because it's shown on a small space while the compressed image could have a very different resolution. How do you compare if those two are perceived the same, no metric, as far as I know factors that in.
00:35:50 Jake: Yes. Agreed and it's-- but I think some of the studies that the JXL folks have done have shown some of these metrics that are supposed to be somewhat close to human perception are not whereas I think there's some of the new ones the [unintelligible 00:36:07] is that how it's pronounced? [unintelligible 00:36:09] I think that does [unintelligible 00:36:11] I think that does much better.
00:36:15 Surma: Yes. That is one of the best psycho visuals we have, it's also quite slow to calculate, which is, I think why hasn't seen a lot of adoption, but again, I don't think it factors in density or display size or anything like that because the artifacts can be quite big in a way, but if it's this, but in a very small scale, it doesn't really matter. I know this is also hand wave, but basically, I agree with you that screen size and screen density are factors for what compression artifacts are acceptable and they should probably be passed in like the whole, like, this is-- I want to compress this image for a 2X display pixel density ratio. Should probably should be parameters for the compressor because it allows different optimizations.
00:37:03 Jake: How would you design a image compression test that was relying on human perception?
00:37:13 Surma: In the end, after reading a bit more about color spaces and some of the things, there's so many factors that are hard to standardize and definitely, even your environmental lighting affects how you perceive an image. It matters whether you look at it on a screen, in a dark room or in a lit up room all the-- nd by the whether it's sunlight or warm room lights or whatever, all these things affect your color perception. In the end, I guess you would have to have a study with a huge amount of participants and you record the things that you can control for.
00:37:56 Jake: What device are they on. Stop them from zooming in.
00:38:02 Surma: Will we, at some point, for example, get a media query that allows us to detect whether night shift is on or what color temperature, is the screen auto corrector, or there was a huge effect of how the image is perceived. Is that something that needs to be part of immediate query and a codec parameter? I'm not sure, but that's what--
00:38:24 Jake: There's even a fundamental difficulty in if you show the user two images and you ask them, which is best, it leaves the door weirdly open for a compressor that is creating a more pleasing image, but it doesn't represent the originals.
00:38:46 Surma: Yes. Do a bit, well, [crosstalk] add some clarity, add some texture, edge detection, increase the edges bit of contrast. That's what I've been noticing more and more. When you take a photo at least with the pixel phones and you immediately open it up, it is still processing and if you look at what happens once it goes from the preview to processing, the phone does a lot, like they actually, they do all kinds of smoothing and contrast enhancing. It's really interesting. I guess that's what you mean, like if a Kodak would apply those things, that could skew it.
00:39:27 Jake: Yes. Even just like give someone smoother skin so they look younger. Is that better? I think what I've seen some of the recent Kodak comparisons do is they give you a keyboard shock or a button that lets you toggle to the original but I think that presents its own problems because I think that highlights the artifacts in a way that the human eye would not have always detected.
00:39:54 Surma: That's a very tricky thing because I feel like what you aim for, with image compression on the Web, at least, you want to create an image that looks right. It doesn't need to be indistinguishable from the original. It just needs to look right.
00:40:09 Jake: It can misrepresent the original in a way that would-- if it's a product image, now misrepresents the product. It's not got some of the-- it's turned a rough edge into a smooth edge. Whereas when the thing arrives and it's got a rough edge, it's like, "That's not the picture on the website."
00:40:26 Surma: Right. If one edge is slightly less sharp in the background, which is not essential, or a certain gray is slightly desaturated, that will stand out if you do this flicking back and forth or you have them side by side. In the end, is it a valid and good compression? If those changes allow you to reduce the image by 10%. That's unlikely, but let's assume that will be the case.
00:40:51 Jake: Yes. I think it's still right that you would be picking between two compressed images and you'd be picking which one is best, but yes, I think there would need to be a button that would show you the original. I think the key there is that it doesn't show you the original in the same X and Y position as either of the compressed versions. You can imagine a split view where on the left-hand side, you've got one compressed version, on the right-hand side, you've got the other compressed version, but when you do the toggle to the original, it's showing you the original in the middle.
There, because it's got that horizontal shift, it no longer gives you that superhuman perception of what has actually changed. I think that would work because you would be able to see, "Oh, hang on. This product has changed." Your human eyes would notice that because you have, as a human, decided that that's the focal point of the image. Whereas, yes, if it was something in the background that wasn't quite as sharp, your human brain is like, "Well, I'm not looking at that so hard. That's not the focus of the image." If that is a little bit different, who cares? I don't know. [crosstalk]
00:42:00 Surma: I wonder how many people we've lost over the last couple minutes by nerding out on the details of image compression.
00:42:08 Jake: Oh, well, Cassie Evans for one.
00:42:10 Surma: Oh yes, for sure.
00:42:10 Jake: This would be a massive waste of her time.
[laughter]
00:42:13 Surma: Because, for the first time, I'm recording this in GarageBand, I only know that we've been recording for 1,550 bars.
00:42:25 Jake: Excellent.
00:42:25 Surma: I've no idea what time we're at.
00:42:28 Jake: One thing I wanted to talk about is, I guess I've been recording two or three with other people, and it's been weird going from a situation where we did it for years, you know? It was all kind of--
00:42:41 Surma: A well-oiled machine.
00:42:43 Jake: Well, yes. Certainly, there was lots that was just second nature or that we did without thinking about it, that when you bring someone new into the studio, you have to make sure they know about these things that you've just forgotten are even weird. One of those things happened with Ada on the recordings I did with her. Talk me through how an episode of two or three ends in the studio.
00:43:07 Surma: Well, I feel like you get a sense for when there is no more slides and you have the ominous, "Ah, I guess that's how you do keyboard events," or, "This is what I learned recently about Deno," or whatever. It's like the subtle phrase about, "This is the end, mate," and then we just stare at each other awkwardly for about a minute and just hold it because we need a bit of tape for the fade-out and the credits.
00:43:44 Jake: Exactly. It's weird, isn't it? It's not a minute. It feels like a minute. It's about five seconds, I would say, where as you say, "That's how we do keyboard events," and we just stare at each other. Then-
00:44:00 Surma: [chuckles] Longingly.
00:44:01 Jake: -someone will go, "Cut," and then we're done.
00:44:03 Surma: Then we all just relax into our chairs and recover from the awkward eye-locking that we had to endure.
00:44:09 Jake: Exactly. I didn't tell Ada about that.
[laughter]
I just totally forgot. What would happen is I would do as you said and say something like, "By doing that, we can make the Web faster," and Ada went, "Yes." Then I'd stare at Ada, which in normal human interactions-
00:44:35 Surma: Means, "Say something."
00:44:36 Jake: -is, "Say more. Fill this empty space."
[laughter]
Because that's what you do with awkward silences. That's what would happen, like Ada would for a couple of seconds, and then go, "And, and-- Well, another thing--" We record an episode for about 40 minutes.
00:44:54 Surma: [laughs] Well, the thing is that Ada is also developer advocate, and quite good at talking and probably-
00:45:02 Jake: Knows loads of stuff.
00:45:02 Surma: -felting that needle.
00:45:04 Jake: Absolutely. There was [chuckles] a hell of an edit job on one of the episodes, just because we covered so much. The content was good but I didn't think people were going to sit through 40 minutes of it. I think we edited it down to 20, but it was absolutely brutal. That was in my notes now, right at the top, was, "Remind people about the awkward end."
00:45:32 Surma: Did it work with Cassie then?
00:45:34 Jake: Yes. Oh, it was first thing I said, by the way.
00:45:36 Surma: [chuckles] This is going to be awkward.
00:45:38 Jake: There's going to be a bit where I say, "Now we know about SVG paths." Nods. Nod. Nod. Nod. Nod.
00:45:44 Surma: Should we give people the inside scoop about the original first episode with someone else than me?
00:45:54 Jake: Yes. What is it?
00:45:57 Surma: We were chatting. At some point, you were like, "Oh, I'm going to try and do now rotating guests on 203. I'm going to start with Ada." I was like, "Oh, that's a really cool idea. Looking forward to it. Blah, blah, blah." Then at some point, you sent me DM saying, "Would you be okay if there was a weird Surma puppet that we are worshiping?" I was like, "Yes."
00:46:22 Jake: The mannequin. Oh, yes. I was upset that I had to send that message to you.
00:46:28 Surma: I know.
00:46:30 Jake: Our production team made me send that. Because I thought it would have been so much funnier that if you were seeing the-- for people who haven't seen it, at the start of the first episode I did with Ada, I thought it would be funny to do a little sketch as if I had created a mannequin of Surma in order to--
00:46:48 Surma: Grease my loss.
00:46:50 Jake: Exactly. Yes, to help me get over you so I could sit in the studio with you once again. Ada walks in and sees the mannequin, and I'm like, "Oh, nothing to do with me. Ha, ha, ha." I pitched this idea. They went for it.
00:47:06 Surma: You did film it.
00:47:08 Jake: We did film it and it took all morning. It made me realize, because we film episodes of 203 just as-- it's one take and done, and then we edit it. Whereas it turns out if you're doing a scripted thing with walking, and talking, and props, that takes all morning to do what was a one-minute sketch. It was an absolute nightmare. Yes, they said it like, "Are you sure Surma is going to be fine with this," and I said, "Yes." They said, "Have you checked?" and I said, "No, but wouldn't it be funnier?" They went, "No, that would not be funny. You have to go and talk to him." Okay, fine.
00:47:47 Surma: I understand them, but I agree it would have been a lot better if I hadn't known about it. That would be really good.
00:47:54 Jake: Oh, yes, we have talked a lot. Wait. It feels like, again, didn't talk through half of things I wanted to talk about.
00:48:02 Surma: Well, we had a lot of time to collect topics to talk about, so I think we'll be set for a couple of episodes.
00:48:10 Jake: Yes.
00:48:11 Surma: Should I jinx it again? Maybe we'll do another one in a shorter period than half-a-freaking year?
00:48:19 Jake: Yes. Let's give that a go. I mean, it's over half a year, isn't it?
00:48:22 Surma: Yes.
00:48:22 Jake: That's the terrifying thing. Yes. From February until now, I've just been bouncing from deadline to deadline, which is really unusual in this role for me.
00:48:36 Surma: The video did on the page transitions did do well, didn't it?
00:48:40 Jake: Yes. Oh, actually, do you know what? How long is this podcast going to be? There is actually-- an interesting element to page transitions is that the default animation that happens of the-
00:48:55 Surma: The fade?
00:48:57 Jake: -the fade. It's not just a fade because it also use-
00:48:59 Surma: The transition effect.
00:48:59 Jake: -the element from it, but that includes width and height.
00:49:06 Surma: Interesting.
00:49:08 Jake: Yes.
00:49:09 Surma: Does that mean if I have a diff with a line of text in it, and I have the same diff that is a lot higher but with the same text in it, that in the transition the text will get stretched, because they're flattened to a texture, right, during the transition?
00:49:25 Jake: Yes. The defaults we're going to do is the two images, the before and after, are going to sit at the top of the box and match the width. You can think of it as like, if you put an image in there in an element and make it a 100% width, it sits at the top of the element and scales with the width. We went through a lot of the material design-stuff on transitions, and that seems to be the sensible default. You could apply height 100% as well and correct stretchy thing if you so want.
00:50:03 Surma: Basically, it has an aspect ratio by default of the original aspect ratio?
00:50:08 Jake: Yes. Height, auto, width, 100%. You could give it height, 100% and it will be stretchy but then you could apply objects. Object fit an object position. You can customize it that way. We're animating width and height, which puts us in traditional terms, the main thread path. It's the sole little layout in the layer for these-
00:50:35 Surma: Yes, because there's only the pseudo-elements, which there's three. [laughs] If you go nuts, there's 10.
00:50:44 Jake: Right. It definitely puts us on the main thread, and that would be problematic for regular NPA navigations because there's a bit of a main thread car crash when you take it slow.
00:51:00 Surma: What's interesting, because I suppose in this specific scenario, there is no layout, everything is positioned absolutely.
00:51:08 Jake: Well, there's layout if you're changing the width and height of one thing, and the thing inside it is width, 100%. It's still layout.
00:51:17 Surma: I guess the hierarchy still exists. I wasn't sure if they're still hierarchical of everything just goes to top-level flat, just the top levels of children.
00:51:25 Jake: No. We have nested pseudo-elements. We've got a container and then we've got a wrapper, and inside that wrapper, we've got the before and after images.
00:51:37 Surma: The reason I thought that is because you had the example in the video where with overflow hidden, things animating from inside the container would get clipped, and now in the patronization API, that wouldn't happen anymore. I thought that was because they all get promoted to top level.
00:51:57 Jake: They do, but every transitioning thing is a nested structure that is the container, the wrapper, and then the before and after images. One feature we do want to add is the ability to nest one transitioning element within another for the cases where you do want that. I don't think that's what we're going to ship initially. Yes, we still have nesting of the pseudos, but each transitioning element is top-level, but each transitioning element is made up of a structure of pseudo-elements.
That's fun as well, because it's not the first time that's happened. You can have a marker inside a before. Marker, that colon, colon marker which is for bulleted lists. That's the marker. Yes, you can have a before with a marker. I think that's one of the only other nested pseudo-elements. We're jumping in with three levels of nesting and multiple children and stuff. It's still quite a big thing.
One of the things I've actually been trying to figure out today is if our selectors are doing the right thing, given that that's what our pseudo-elements are doing. The thing I wanted to say is because we could do these transitions using pure transforms, but we decided not to because that makes them really hard to customize especially if you want clipping because you end up with the counter transform thing happening, all of that sort of stuff.
We said, "Let's just animate this with width and height because that is the simple thing to do. Developers understand it. It's easy to customize." Our next mission is, can we move it to the compositor for these specific cases? How much can we stay on that happy path with the likely customizations that developers are going to do?
00:54:02 Surma: One of the big advantages is that the animations are declarative.
00:54:07 Jake: Yes, but if they take the contents of one of those images, and makes it text, which you can do. Now you've got the text layout inside your transforms. We'll have to bail. There's no way we can do that on the compositor.
00:54:26 Surma: No, for sure, but also it's declarative. These things can be statically analyzed and we could put warnings in Dev Tools. I say we, it's not we anymore, it's you.
00:54:36 Jake: Now it's me, isn't it? That is the plan. Because if you add a border to the transitioning elements, again, I think that's where we start to bail out because we're not going to be able to do that entirely on the compositor.
00:54:49 Surma: It seems like an awful idea, visually.
00:54:52 Jake: Isn't it? It depends. If you've got a button that has rounded corners, and it changes with, one of the things we were thinking of is, can we do this in a way that it stays on the compositor side, but also doesn't stretch the image. There's all kinds of fun things we're thinking about, but for that. It might turn out that the transitions are fast enough that it doesn't matter in most cases, but again, stuff to figure out.
00:55:22 Surma: It's going to be fun to test it on low-end phones. Get the banana phone back out.
00:55:27 Jake: Yes. One of the ideas we threw around was that we just pause everything during the transition so we don't get that car crash at the start of a page loading. I think that the opportunity of page transitions is that you'll be doing a lot of work while the page transition is happening. We've got a lot of comments, stuff like, "Oh, this is going to make pages slower." It's like, I hope it wouldn't because I hope that the transition happens but in the background, it's still setting up the stuff for the first interaction.
00:55:59 Surma: I wonder if the layout could happen on a different thread, because there's no user JavaScript or anything in that moment, that the layout for just the transition elements could be a separate thread.
00:56:12 Jake: That's one of the other ideas we've had. We are shied away from that for now just because that would be such a major re-architecture of how the browser works.
00:56:23 Surma: That'll be a reject for sure. I'm guessing it would be similar to a work-light architecture, but I can understand if the MVP of this API doesn't do that.
00:56:34 Jake: Also, the CSS that is driving this stuff is on the main thread. We'd be shipping that stuff across, and what if it changes halfway through? It is one of our possibilities if we can't do it any other way.
00:56:49 Surma: You know what, we should really talk to Cassie about this.
00:56:52 Jake: This is animations. Why isn't Cassie here? Oh, now we've wasted her time. Do you know what, she's probably talking to someone about coffee or something, which is a massive waste of her time, when she could be using her time valuably right now? What a shame?
00:57:10 Surma: Such a bad man. I think that that is a good point to just call it before we descend into further [unintelligible 00:57:21].
00:57:22 Jake: Absolutely. Absolutely. Well, this has been fun. Hopefully, get this edited and out sooner than six months. I guess, there's nothing else to say. God, we haven't done this for a while. Nothing left to say, except-
00:57:39 Both speakers: Happy next time.
00:57:41 Jake: Bye.
[music]
Is the pair programming app Surma talked of https://tuple.app/ ?