Hello, my friends. It’s (Q2) of a (not-so) new year. That means it’s time to talk about Panic!
Here’s what we did in 2017, and where we’re going in 2018.
Releases
Transmit 5
2017 was all about Transmit 5, our long-awaited major update to your favorite truck icon.
It had been an astonishing seven years since we had done a major update to our file transfer app for macOS. We’d spent most of that time focusing on Coda, so we had a long list of things we wanted to address, and once we got started it took us a good couple of years to get it all done. The good news is that we landed virtually everything we wanted to land in 5.0. And we focused hard on quality and process, creating what I think is the most solid initial release of any app we’ve ever done, ever.
The improvements in the app were huge: speed was increased significantly. Eleven new cloud services were added. We added our fabled Panic Sync service, which remains free and secure. I’m really happy with our “Get Info Sidebar” and I wish it was in the Finder itself. We added a Batch Renamer. We built a great UI for managing private keys. I could go on and on.
You can see all of the new features over here. And we made a fancy little video to show off what’s new:
[youtube https://www.youtube.com/watch?v=dwtaJRWvuSU&rel=0&w=750&h=422]
After shipping, we naturally collected feedback and we addressed most requests in the just-released Transmit 5.1, including support for SFTP’s “ProxyCommand” config setting, support for Amazon GovCloud, and brand-new French, German, and for the first time ever, Chinese, localizations.
How’d it do, then? What’s the business side of macOS productivity software like? I’m happy to say Transmit 5 sold quite well, all directly through our store. And the timing was (accidentally) totally great — just as Firewatch revenue was starting to wane as the game ran its course, Transmit 5 fully picked up the slack. It was comforting to see that people are still willing to pay real money for good Mac software. That’s why we’re here and can continue to do what we do.
One interesting note: We added a ton of new cloud services in Transmit, which is great, but it definitely adds an exponential burden to our QA process (many connection types × many functions = many many tests). Interestingly, to this day, still nothing beats the popularity of plain old SFTP, and, oh my stars, FTP. Yep, regular, unsecured FTP. Based on our anonymous usage data, that unnecessary donut chart on the right there shows how the connection types rank so far.
The purple and dark blue? That’s everything else – all cloud services combined. Were they worth adding? I think yes, as a bet on the future, but we’ll probably want to pick and choose future protocols extremely carefully.
Overall, this release had an extremely low goof-up ratio, but there were a few pain points. Our simplified activity window redesign was much cleaner and clearer for the people who just wanted to know what Transmit was doing, which we think is most people, but lost some really important functionality for users who need to manage their pending transfers more granularly. We’ll keep working on this in future versions. And some were disappointed that we didn’t provide an upgrade discount for our existing Transmit 4 customers. Normally we would’ve, except for that seven year gap — we’d been providing free updates for seven years, and that seemed like extremely good value for money? (Interestingly, many of the people upset by this had been using it for all seven years!) Finally, a couple people really couldn’t handle the new Sync toolbar icon! These things happen. There’s a reason it changed: it used to be 
, but now it’s 
, as the new 
button uses arrows, as well as the usual 
button, and it was just too many arrows!
But overall, it was extremely smooth sailing. And we’ll keep working on improving it even further, of course!
All of us at Panic would like to say thank you to everyone who bought Transmit 5. And thank you to everyone who sent us their ideas for Transmit 5 and beyond. I’d also like to say thank you to everyone at Panic, including of course Wade and Will who spearheaded this release, for doing such amazing work.
Faster?
We often write things like “we made it faster!” without going into much detail — that’s usually because the details are pretty gory. But, exclusively for this update, I thought I’d ask Wade, who did the bulk of the work in Transmit 5 with his brother Will, to get specific — what did we do to make Transmit 5 faster?
For those interested, let’s get technical!
////////
// // P A N I C
//////// E N G I N E E R I N G
//
When we sat down to start work on Transmit 5, we wanted to prioritize speed and the “cloud” (have you seen all the new cloud services Transmit 5 supports?!). While adding cloud services was fairly straightforward, improving speed can be a little more abstract. We approached the problem from the ground up and found obvious areas where we could improve and some that weren’t so obvious going into it.
The first and most obvious change we made to Transmit was how it handled transferring large file hierarchies. In prior versions of Transmit, when you started transferring a group of files and folders, it would transfer the top-level items using individual connections to the server, but the folders and their contents would be processed serially using one connection. This worked well if you were transferring many top-level folders, as they would be transferred concurrently, but transferring a single folder would only ever use one connection. With today’s fast internet connections and multi-core devices we felt it was time to fully multi-thread every possible transfer scenario. In Transmit 5, almost all transfers will use the maximum available connections regardless of their contents. (Some cloud services restrict the amount of API calls you can make to the service over a period of time, so those services are limited to two transfer connections.) This allows multiple connections to work on the contents of a single folder, which results in much faster transfers.
This in and of itself was a huge improvement, but we didn’t stop there.
Long-time Transmit users may remember that in Transmit 4 we introduced our patented (not really, but it’s still cool) dual progress bar. This style of progress representation shows the entire transfer progress as well as the current file’s progress in a single bar. While this was great for showing the user status about their transfers, the necessary pre-flighting to compute the entire transfer size was delaying the start of the transfer. In some cases we found the entire transfer could be completed in the amount of time it was taking Transmit just to compute the overall transfer size! Transmit 5 now computes the overall transfer size dynamically and asynchronously with the transfers themselves, so your transfers start immedately and you still get that sweet double progress bar.
Once we got the backend completely multi-threaded, we noticed that, under high stress transfers, the app could start to feel a little bogged down. Turns out, when you transfer thousands of files in a few seconds on gigabit ethernet there are a lot of messages being sent around internally. Folks who know programming know that you have to process user interface updates on the main thread, but that thread also handles user input, so if you’re overwhelming that thread with messages, your app can start to feel a bit sluggish. This was happening to Transmit, as it was servicing progress updates constantly. We developed a lightweight messaging queue that gathers updates from the connection threads, coalesces redundant messages, then batches the delivery to the main thread to update the user interface. The coalescing reduced many unnecessary progress updates and batching allowed us to fine tune the amount of work that the main thread would perform per update cycle. Likewise, we added a second layer of message throttling from our various connection classes to the message queue, which further reduced the amount of internal messaging while maintaining a high level of performance and responsiveness.
We left no stone unturned in our relentless pursuit of improving all aspects of Transmit’s performance, and memory usage was certainly included on that list. After all, your computer isn’t going to feel fast if memory is paging to disk all the time. We analyzed the memory consumption and to our surprise, NSScanner was using a very large amount of memory when parsing large directory listings. I wrote an API-compatible replacement that instantly eliminated the memory bloat and had the side benefit of being faster as well. That’s what we in the industry call a win-win. Beyond those parsing improvements, we restructured the way very large file transfers were handled to prevent temporary memory bloating as well. This means you can transfer multi-gigabyte files using Transmit and memory use will remain stable.
After the backend systems were performing at a high level, we turned our attention to the UI. Frequently drawing and animating progress bars can require a fair amount of CPU. In Transmit 4, when you had the transfers list visible with several transfers in progress, the CPU usage would tick up a few points (which our sharp-eyed users were quick to point out). Lucky for us, a few things happened in the computer world since Transmit 4 came out. User interface design trends are now much simpler and flatter, requiring far less work to draw. Additionally, highly performant layer-based drawing is better supported. We rewrote our progress bars to be CALayer-based, which reduced the CPU overhead tremendously. Those optimizations keep the main thread free to handle other tasks that keep the app feeling snappy.
All of this work resulted in massive speed improvements across the board when using Transmit. Not only are transfers dramatically faster, but the entire app benefits from the internal work across its various components. Occasionally while working on Transmit I’d think something was broken because the transfers would complete impossibly fast! It wasn’t just us that noticed these improvements either, we immediately heard from users about how much they were loving the speed of Transmit 5. -Wade
So when you see a single bullet point in a major Panic release that says “• Now faster!”, now you know.
Updates + Support
I asked Aaron, who handles a great deal of the QA and release process here at Panic, to talk a little bit about what we did, software and support-wise. He says:
In 2017 we shipped a record 39 software releases. Each one of those releases had to be qualified, tested, approved, and distributed. So, for our small team, this is a really solid accomplishment. Here’s what went out the door:
|
|
|
|
|
|
|
| 2.6.1 |
2.2.4 |
2.6 |
4.4.12 |
5.0b1 |
1.3.2 |
4.4.13 |
| 2.6.2 |
2.2.5 |
2.6.1 |
4.4.13 |
5.0b2 |
1.3.7 |
5.0 |
| 2.6.3 |
2.2.6 |
2.6.2 |
|
5.0b3 |
1.3.8 |
|
| 2.6.4 |
2.2.7 |
2.6.3 |
|
5.0b4 |
1.3.9 |
|
| 2.6.5 |
|
|
|
5.0b5 |
☠? |
|
| 2.6.6 |
|
|
|
5.0b6 |
|
|
| 2.6.7 |
|
|
|
5.0b7 |
|
|
| 2.6.8 |
|
|
|
5.0b8 |
|
|
| 2.6.9 |
|
|
|
5.0 |
|
|
|
|
|
|
5.0.1 |
|
|
|
|
|
|
5.0.2 |
|
|
|
|
|
|
5.0.3 |
|
|
|
|
|
|
5.0.4 |
|
|
|
|
|
|
5.0.5 |
|
|
Improved QA
2017 also brought a number of improvements to our internal QA processes and organization.
We retooled our testing documentation using new features introduced in GitLab and formalized a new release workflow to better coordinate our Portland and Japan offices.
The Transmit 5 development process also resulted in the creation of new internal tools for testing and verification:
Fig • Quickly build, snapshot, and deploy macOS configuration profiles
Hydra • Panic’s authentication and file transfer testing system
Pug • Quickly update local projects and their dependencies with a single command
Vanderbilt • Browse and download builds of Panic apps using the command line or GUI
We’ll spare you any overly specific details for now, but if people are interested we could do a more extensive write-up in a future blog post. Let us know! ?
Improved User Help
In addition to all of the fun new tools we used internally, there are also some helpful new user-facing resources we’re really excited about. First, we introduced a dynamic Panic Help Library inspired by Acorn 5’s Live Help Search. (In his write-up, our pal Gus of Flying Meat said “If you make a Mac app, please steal this idea”, so, like, we did! But no, if he jumped off a bridge, we wouldn’t.)
Here’s what our new live help looks like in our app:
All of those help articles are being pulled from our server in real-time when you search! That means we can add new features or documentation at any time without needing to push an update to Transmit itself, and as a result our help is ever-evolving and ever-helpful, and everybody wins.
Tutorial Videos
Right near the end of the year, we debuted the first of our Transmit 5 tutorial videos. For years we’ve been wanting to show off cool tips and tricks in our software in the most visual and dynamic way possible, but didn’t exactly have the time or bandwidth to make it work until Christa came on board. Now we’ve got video coming out of our ears! And eyes. And computers.
Our videos can teach you about how to best use Places…
[youtube https://www.youtube.com/watch?v=i6He9Upze7c&rel=0&w=750&h=422]
…or how to use our File Sync feature…
[youtube https://www.youtube.com/watch?v=S7FSlGsJtZc&rel=0&w=750&h=422]
…and lots more. We’re adding these frequently! Consider subscribing to our YouTube channel! Yeah, I said it.
And a request: if you have topics you’d like to see covered in a future video please let us know. We have a whole lot of exciting new things planned for 2018!
This is the real engine that keeps Panic moving and software going out the door. Our process continues to improve significantly year-over-year. Also I like that Aaron uses emoji in his updates.
Services
It’s a huge part of our business we don’t discuss much: our backend services that help you sync your data, process your orders, and the like. Here’s Patrick with a brief summary on our service work for 2017:
The site had numerous under-the-hood changes to support a big internal effort — modernizing our web store.
For years (almost 20 years, in fact) our web store was powered by a custom PHP app we wrote called POD, or the Panic Order Database. It grew a lot over time, to handle sales of physical goods, product activations, Coda plug-ins, and all kinds of other things. It became a little long in the tooth, and in 2017, seeking a more flexible and modern architecture, we decided it was time to retool.
Building off our great experience with Django in Panic Sync, we starting rewriting our order system as a Django app, cutting out slices of functionality and migrating them over bit-by-bit. In a true Ship of Theseus fashion, almost all backend pieces have now been replaced by their Django counterparts – now complete with tests, code coverage checking, Python 3.6 type hinting, and other great tooling improvements.
Also, Panic Sync 2.0 shipped, the headlining feature of which was 2-Factor Authentication. We added a huge number of new users quickly – over 10,000 in a very short period of time. (And speaking of big numbers and weird backend services, our firewatch.camera project is still running strong, currently hosting over 680GB of in-game photos across over 54,000 rolls.)
We’ve standardized on Django across all our web services and we couldn’t be happier with it. Python 3.6 and Django 2.0 offer a great developer experience, modern tooling, and years of stability. Hey, if it’s good enough for Instagram, it’s good enough for us!
This work is always ongoing and represents a huge part of what we do.
Infrastructure
A bit more from Patrick:
At the risk of sounding corporate and corny, we’d like to say that independence has always been a core value of Panic. As such, it should be no surprise that we think of our web services in the same way: while we believe companies like AWS, Heroku, and DigitalOcean all provide valuable services, we still prefer to build our sites on our own servers and infrastructure – the same way many of our own customers do. We have the know-how, gumption, and by-golly, the stick-to-itiveness [this is gonna be tough for Noby to translate into Japanese! —Ed.] to do it, so why not?
Over the past year, James has been busy upgrading the hardware that we have colocated at the Pittock Building (a few blocks from Panic in sunny Portland, Oregon) significantly, as well as rebuilding most of our servers to run on shared, virtualized host clusters, set up and managed by Ansible. Using Gitlab, we now have continuous deployment for all of our sites, keeping us in line with modern web development best practices.
These improvements have proved to be so fast and stable that in 2017 we moved all of Panic Sync off Heroku and onto our own infrastructure. For the paranoid among you – don’t worry. Your data is still safe: you can read more about how we encrypt your data so even we can’t read it on our Panic Sync page and library article.
Speaking of security, we’ve also upped our encryption game internally: all source code interaction now requires 2FA, and we’re requiring Yubikey or Krypton for private key storage. We’re thinking of doing a separate post from the engineering department to dive a little more deeply into our security setup, so if you’re interested in that, please leave a comment and let us know (and don’t forget to smash that like button!) [There is no like button —Ed.]
If you ever have questions about this stuff, let us know!
Successes
So, looking back, here’s what I was proud of from 2017:
- We still ship pretty nice software. I mean, we’re definitely not perfect, but we’re making really quality things, I think. Transmit 5 modernized the UI, added a ton of stuff, increased speed, and was an extremely solid release that a lot of people bought. As long as we can keep doing this well, we’ll hopefully be here for a while.
- We’re still having fun on the web. The Transmit 5 website is, I think, really nice, especially thanks to our WebGL truck magic. We worked with Little Workshop (look for a blog post about that in the future!) who took Kenichi’s 3D icon and make it work on the web. (They were also not afraid when we said “and also the perspective of the truck should change when you scroll!”.) And thanks to Dean at Apple on making it extra performant in Safari. I think it’s a good sign when your crazy product web page leads to overall improvements in Safari for everyone.
- Our support is better than ever. In addition to the improvements mentioned above, including our Panic Library, I think our response times are fast, our answers are accurate and helpful, and people in general hopefully feel that when they buy something from us, we have their back. (And when we fail, we fix it!)
- Oh and we expanded our work area and added a couple trees. It’s almost all done.
Challenges
Not everything was silky smooth sailing in 2017, including:
- iOS, again, as always. We have not cracked the secret of how to be super successful making advanced productivity software on iOS. (In fact, quite the opposite — we canceled Transmit iOS, our portable file transfer client, due to poor sales.) I’ve talked about this for many years running, but I’m hopeful this is the year we’re gonna crack it. In the meantime, we’re still dedicated to updating and improving Prompt and Coda, our iOS apps that can justify increased investment.
- We got owned hard. We blogged about it at length right here. It wasn’t exactly our favorite time. But I can give two small addenda to that story: we never heard a word from the attacker after we threw all our cards on the table via that blog post… and the law enforcement investigation is, amazingly, still ongoing.
- We don’t have enough engineers. Our dream of “an engineer permanently assigned to every app” continues to be strained with our current, rather intense workload. And low iOS revenue doesn’t give a lot of room for growth. There’s a lot of juggling to keep everything updated. But, it’s time to staff up. Read on for more about that.
- Our process still has room for improvement. Defining features and releases more concretely, and putting more time into UI up-front before code gets written (including building in time for design prototyping) so that UI designs are ready and more thought out earlier, are just a few of the things I hope to see get better.
- Tough revenue decisions. We continue to debate about how much longer people will be accepting of full price macOS software. Is it time for us to consider subscription pricing, which could lead to frequent and constant Panic software releases, instead of saving up for big “major updates”, which sounds really appealing? On the other hand, we know a lot of people don’t like subscriptions, for many valid reasons. We’ll have to do more research on this throughout 2018.
Q&A
I asked on Twitter if anybody had any Panic questions. Here are ten of them!
① Do you have plans for Prompt on macOS? —Lukas and Josh and Mike etc.
We talk about this idea a lot! While we’re not working on it right now, I’ve always thought it’s a good idea…
② Are you involved at all with Campo Santo’s next game? —Michael
We’re not. You may have heard that the team at Campo Santo now all work for Valve, a move that will allow them to focus more on making games and less on running a business at a scale well beyond what we could provide them. Frankly, it’s still a little hard for us, as we truly enjoyed working with them. But, we can’t wait to play their game some day. And soon we’re announcing our second game as a publisher — more on that below!
③ Who’s behind the lovely videos that have been shared recently? How long do they take to make? —Dave
Christa does those! She consults with engineers/support/QA to understand the goal of the video, she writes the script, she does all the After Effects work, even the voiceovers. (I do a bit of copy editing on the scripts and try to write original music for them as much as possible!) A typical video takes… a week?
④ Will you ever make shirts again because some of my favorite shirts are lookin kinda rough ?? —Mike
This is a very good idea. We really should do another run of Panic shirts. Maybe we can pull this off in 2018.
⑤ Any plans to bring Verso to BeOS? —Chris
How… how do you remember this incredibly obscure bit of Panic trivia… Verso, geez.
⑥ You probably get this a lot, but would it be possible to get an office tour some day? —Joan
In general, yes! It depends on what we have going on on a given day, what’s lying around, and if I’m in the office (most people here are a little scared of humans), but it’s always possible. Twitter is a good way to schedule that!
⑦ Can you talk about any projects that you thought of, started, but then abandoned? —kmikeym
We once canceled a very polished and beautiful iOS music sharing app called… Audion. (I’m not kidding!) Kinda like a musical Instagram. It had at least one UI thing that I still think is original and amazing. Someday we’ll tell that story.
⑧ Any update/change about the last status of ? Transmit for iOS? —Fred
We are still working on our plans, starting with: is there a way we can bring more of Transmit iOS’s functionality to Coda iOS? That would be a big change to Coda iOS, but might make the most logical sense. Stay tuned.
⑨ I’m using Transmit 5 as my primary interface to Dropbox. Am I a bad person? —Kyle
Absolutely not. This is so potentially powerful, we even made a video about it!
⑩ Is it just me or is it cold in here…? —Ashur
Ashur, we’ve been over this, it’s hot outside right now so, YES, the cooling kicked in, that’s what office HVAC systems do, they COOL or they HEAT. Maybe our next bonus should just be JACKETS??? [Ashur works at Panic and is also responsible for the Slack channel #planet-hoth. —Ed.]
What’s Next
In addition to maintaining all of our apps, which, remember, we are doing always, here are the big things in store:
Firewatch for Switch
Look for it later this year!
A Second Game
Soon we will also announce the second video game we’ll be publishing! The follow-up to our hit game Firewatch will be a new game from a different studio that is not at all like Firewatch but is absolutely delightful. Keep an eye on our blog and Twitter for the big reveal.
Coda Next
Great news! Our work on the next version of CODA® is well underway. Here are some facts I’m allowed to share about this project.
A lot has changed in the web development world since we first started working on Coda, not the least of which is a new set of really capable (and often free) competitors.
To catch up to today, we had to take a dramatic step. We’ve been informally calling it Coda Next during production. (We may even rebrand the product entirely, since it’s a dramatic step forward from today’s Coda.) It’s a total rewrite of our macOS version of Coda. We’re taking everything we’ve learned from the past, everything we learn doing modern web development here every day, and everything we’ve heard from our customers, and we’re totally redefining Coda for the future. Expect support for cutting-edge development and deployment workflows, ways to make your work easier, a light bloat-free design, and most importantly, 100% macOS native code so it’s extremely fast and responsive. This will be a huge update.
But, how far along are we? We’ve completed work on the “editor core”, i.e. the editor itself, i.e. the real meat-and-potatoes part of Coda Next. We’re using a brand new engine based heavily on Coda iOS, and including much-requested features like multiple cursors and a minimap. We’re really happy with it so far, and many of us are using it internally for our day-to-day editing already.
Coda has always been more than just an editor, though, so now we move on to the rest of the app, which is a lot. Some features may go, some may stay, but everything’s getting a re-think.
I have long ago learned not to predict release dates, so I won’t even try. While work is progressing rapidly, we still have a whole lot to do. But are we excited for you to see what we’ve got? Very!
Hiring
Something extremely rare is happening at Panic: we’re about to bring a few more people on board. Because it’s so very important to our success in 2018 and beyond, I’m gonna slap a <blink> tag on that: we’re hiring soon!
Keep an eye on this jobs page in the next few weeks. (We’ll also tweet when the jobs are up.)
And if you know anyone looking for a nice job in a nice city writing nice software, send them our way.
(My god that blink is really awful, no wonder it was deprecated, sorry.)
Misc
We continue to work extremely hard on extremely bonus projects, as we have for a few years now, and I sure hope we’ll have a surprise or two for you in 2018.
Thank You
I always get a little sentimental writing these posts — borderline trite, to be honest — but it’s really astonishing that we continue to be able to do what we enjoy, day-in and day-out, and that’s all thanks to you. Recently I spur-of-the-tweet offered free Panic stickers and it brought piles of nice cards and letters to our office which really physically drove the point home: we are extraordinarily lucky to have such great customers and fans. Your support of the things we make, be it Transmit 5 or a game like Firewatch or even a Katamari t-shirt you bought a billion years ago, it all has helped us immeasurably. And we hope that we have improved your life a little bit in return.
To keep up with Panic goings-on, follow us on Twitter, or subscribe to our mailing list!
