Push-button Book Publishing with Leanpub
Book publishing used to be a labor-intensive occupation, full of tedium and repetition, but such drudgery could be a thing of the past, thanks to a new system called Leanpub, with which we’ve just published our latest title: “Take Control of Dropbox.” Leanpub relies on a number of key tools — the Markdown language, Dropbox, LaTeX, CSS, and a Web-based interface — to enable a quick and efficient book publishing workflow.
For my earliest books, I had to FedEx final Word files and screenshots to my publisher on CD-R, along with printouts. Those files were laid out in PageMaker, and we pored over the galleys to eliminate errors introduced in layout. At least it was mostly digital — a friend was just telling us about how his first book was retyped (badly) in Ireland, to be entered into some early publishing system.
But you know what? For a single book, that’s not terrible. Publishers got away with such inefficiencies because labor was cheap and the alternatives didn’t scale well. In fact, despite its whizzy features, Apple’s iBooks Author software comes from such a tradition, lacking, as it does, the change tracking and commenting capabilities that are necessary for any professional author-editor workflow.
When we started the Take Control book series nearly ten years ago, we knew we couldn’t afford to play such labor-intensive games, so our authors have always written directly into their final layouts, using detailed style sheets in programs like Microsoft Word and Pages that provide change tracking and commenting for the editing process. We adopted Dropbox quickly when it came out, since it makes swapping versions of a book between author, editor, and publisher trivially easy. When possible, we even take advantage of time zones so we can eliminate some dead time while different people work on the same manuscript.
However, our Achilles heel has always been production, where we create the final book in the most common ebook formats. A lot of that effort goes into making PDF bookmarks and links. Back when we used Word, we cobbled together a system that relied on a squirrelly detour to Word for Windows to generate links and bookmarks. Although Pages makes PDF links, it does so in a long-broken fashion (doubling the PDF link boxes for each link), and it can’t create bookmarks at all, requiring us to use the Debenu PDF Aerialist plug-in for Adobe Acrobat Pro.
Pages can put out an EPUB that looks decent, but it’s so horribly constructed with regard to CSS styles that fixing its EPUB requires a complex production step and a brittle BBEdit text factory. Worst were Mobipocket versions for the Kindle — until our last book, we had our publishing partner O’Reilly Media create them because they’re so hard to make properly. Happily, thanks to a tip from our friend Serenity Caldwell at Macworld, I figured out how to make our own high-quality Mobipocket for last week’s minor update to “Take Control of Using Mountain Lion,” but the process is fussy and complex.
In essence, then, there are two distinct phases of book creation: writing/editing and production. In the first, powerful authoring environments with change tracking and commenting are essential, and drafts need to be moved between author and editor quickly and seamlessly. In the second, the Holy Grail is a push-button system that takes a manuscript and kicks out nicely formatted ebooks in PDF, EPUB, and Mobipocket, complete with clickable links in all formats and bookmarks in the PDF.
I periodically scour the Internet for such systems, and have found several that seemed promising, including Booktype, PressBooks, and Atavist. But when I looked deeper, I’d run into problems. Phase 1 issues revolved around the use of online editing environments that can’t compete with a real word processor, the replacement of change tracking with after-the-fact comparison, and the lack of commenting on selections of text. Phase 2 troubles were related to the inability or difficulty of styling the output as we wanted in each format — the defaults weren’t terrible, but they weren’t
necessarily up to our standards.
Leanpub popped up in one of my searches, and I was intrigued by how they used a variety of existing tools at the surface, rather than building a platform from scratch. The writing and editing in Phase 1 of the publishing process in Leanpub happens in straight text files coded in the Markdown language, with which we’re totally familiar because it’s what we use for TidBITS. (To be precise, Leanpub uses the kramdown superset of Markdown, because the latter isn’t sufficiently rich on its own to represent all the elements necessary in books.) Those files live in Dropbox folders, which we would use anyway, and when you click a button on the Leanpub Web site, it transforms copies of those
Markdown files into PDF, EPUB, and Mobipocket ebooks.
Markdown may be easy to write, but with TidBITS, we simulate change tracking using the Subversion version control system and BBEdit’s comparison feature. And we put comments in HTML comment style within the text. Neither of these approaches is ideal for book-length manuscripts. Although there’s a messy way to compare versions of Dropbox-managed text files using the DropboxDiff Chrome extension, comparing files is far clumsier and more confusing than change tracking, and intra-textual comments clutter a manuscript (and deleting them messes up comparisons even more).
That was when I had a brainstorm — we could write Markdown in any word processor, complete with change tracking and commenting, and then export to text for the Leanpub process! And better yet, since we were planning to test this system with Joe Kissell’s “Take Control of Dropbox,” Joe and I could use Nisus Writer Pro, our favorite writing tool of yesteryear, now back in force. Nisus Writer Pro has full change tracking and commenting capabilities, plus powerful grep-based searching and macros that helped in creating the Markdown. We used character and paragraph styles in Nisus Writer to differentiate various elements of the book
visually as well, which made editing easier.
Even with Nisus Writer Pro, writing and editing Markdown in Phase 1 with Leanpub is still clumsier than working in Pages, because links must be constructed manually each time, graphics must be referenced as external files in a special folder, and things like nested lists with internal screenshots require extremely careful attention to Markdown rules. But it all worked, and it was huge fun to run a Nisus Writer macro that exported the text and then click a Preview button on the Leanpub Web site to generate in-progress versions of the book in each format.
When I found them, Leanpub had just moved from serving only self-published authors to working with small publishers like us as well. They call the concept behind the company “Lean Publishing,” and the idea is to provide free, lightweight tools that make it sufficiently quick and easy to generate ebooks that authors will make in-progress versions available for readers while they write. (We chose not to do that with “Take Control of Dropbox” because we were feeling our way into the system.) To pay the rent, Leanpub acts as a non-exclusive bookstore for the books their users create, charging an entirely reasonable 50¢ plus
10-percent of the selling price per copy. And yes, you can buy “Take Control of Dropbox” through Leanpub for the same price as on our site, though we’re still learning what the overall user experience will be like. (For instance, we just realized that customers on Leanpub can pay more than the cover price of $10 if they wish; a nice touch!)
Thanks to their new focus on small publishers like Take Control, the Leanpub folks are constantly refining and improving their system, particularly with regard to the Phase 2 production step, and it has been a pleasure to work with them. For instance, we were planning to open their EPUB and insert our own CSS into it for custom formatting, but while we were discussing that with them a few weeks ago, they made our lives easier by enabling us to put our own custom.css
file in the manuscript folder and have built into our EPUB automatically. Of course, this benefits everyone who uses Leanpub, but it was still nice to feel acknowledged.
However, although custom CSS improves EPUB and Mobipocket, their system relies on LaTeX for PDF generation, and while the PDF they’re putting out isn’t bad, when push came to shove, we decided it wasn’t something we felt comfortable giving to Take Control readers. In particular, the LinLibertine font was ugly in some versions of Adobe Reader (and Leanpub offers no font family control at all), the font size was too small, and the line length was too long. We also preferred our style of colored boxes behind notes, tips, and sidebars. Alas, Leanpub couldn’t fix all this just for us.
Leanpub does have a solution to the desire for a custom PDF, but it’s not yet ready for prime time, nor are we ready for it. Along with kicking out PDF, EPUB, and Mobipocket (plus HTML, either as a single file or one file per chapter), Leanpub can create InCopy Markup Language files for the book. InCopy Markup Language is a story-specific subset of InDesign Markup Language, an XML-based format that represents an entire InDesign document. The idea is that we could place these InCopy Markup Language files in a laid-out InDesign template, where they’d pick up our pre-defined styles. Changes to the source files could be automatically reflowed in InDesign. We would essentially be using InDesign as an engine for PDF creation. InDesign
could also theoretically export EPUB and Mobipocket, though Leanpub’s versions of those — with our custom CSS — is good enough already. Unfortunately, Leanpub’s export isn’t creating usable files right now, and even if it were, we haven’t yet designed an InDesign template file into which we can flow them.
So for “Take Control of Dropbox,” we decided to fall back on our traditional process for PDF. A quickly generated Nisus Writer macro stripped the Markdown formatting from our manuscript and made sure that everything was assigned style names. Then we opened Nisus Writer’s native RTF file in Microsoft Word, saved in Word format, and imported it into Pages, where our Take Control styles took over. (Other approaches, such as opening the RTF directly in Pages and exporting .docx format from Nisus Writer, didn’t work properly.) Tonya put the internal and external links back in manually, and placed and resized the graphics, adding several tedious hours to our normal
production process. Extra work this time but it worked fine in the end.
We took the opportunity to play with the interior design a bit too, since the other new part of “Take Control of Dropbox” is its use of our new cover design, created by Sam Schick of Neversink, using a logo designed by Geoff Allen of FUN is OK. (The cover colors will vary from title to title.) The new logo and cover are the first steps in a major visual redesign for Take Control. We made some small changes in the book interiors to match the cover color and address some things that we’d been wanting to try for a while — notably, we’ve hidden most external URLs, since they’re long
and ugly in running text, differentiating them in the PDF from internal navigation links by a light underline. Once we’re certain of the tools we’ll be using — Pages or InDesign or something else — we’ll look harder at the interior design. (The final part of the process is a new Web site design, and while the look is done, a lot of ExpressionEngine code needs to be updated and modified, and that’s taking a while.)
We are planning to work our way into Leanpub slowly, since authors and editors have varying amounts of experience with Markdown and Nisus Writer Pro, so we won’t be using them for every forthcoming book. But Leanpub is the most interesting development we’ve found in the book publishing world in a long time, and we’re looking forward to the time when the production phase of the publishing process truly does come down to the click of a button.
Very interesting -- have you considered using Scrivener as the text processor, as it can use MultiMarkdown for text generation? Markdown is essentially a subset of MMD. Better stil would be if Leanpub allowed the use of MMD authoring.
Most regrettably, Scrivener doesn't have the change tracking and commenting tools that we need in order to work collaboratively and rapidly. It's a great program, though, and there's even a Take Control ebook about it.
Hmm. I wonder if we could stick a Pages or Word file, or NisusWriter into Scrivener to get the Markdown output. We'd be using Scrivener only as a production tool, at that point.
Unfortunately, it seems that Leanpub uses "almost all of" kramdown http://kramdown.rubyforge.org/ Markdown superset processor and not basic Markdown as the text parser, so the Scrivener suggestions isn't going to work very well.
Yes, Markdown by itself really isn't rich enough of a language, so the kramdown extensions are necessary for many common book elements.
Fascinating reading about your production process and solutions! You should take a look at Inkling Habitat, which is trying to solve similar problems (disclosure: I'm in the Inkling Habitat team). Check it out and tell me if it helps you with what you are doing: http://inkling.com/habitat. We are constantly refining it based on feedback.
Best,
Brad Neuberg
Twitter: @bradneuberg
I've looked into Inkling Habitat a bit, but haven't had a chance to really explore all it can do and if it fits into how we like to write, edit, and publish books. It's on the list for when I get a little free time!
I freaking love the new cover design! Sam and Geoff did an awesome job.
Very exciting. Now I just need to finish that sequel to Wool that I hope Wool's author doesn't hear about (but sells 3M copies...).
Sorry, but unlike Josh above, I don't like the new cover; it looks like the generic covers on free books from the iBookStore. I feel the traditional cover stands out more and says "a Take Control book!"
Can't please everyone all the time! :-) The need to have readable text at thumbnail size is forcing most ebook covers into a relatively similar look and feel, since there just aren't many possibilities. And because we need to have a series look as well, it's not surprising that our covers are somewhat along the lines of the ones that Apple generates algorithmically for books without specific covers. But I can say with assurance that we were in no way attempting to copy what Apple did - it simply didn't come up - and when our covers are viewed at full size, the differences are clear.
Adam, I understand what you're trying to do, but I haven't had a problem reading the titles on the previous TCo... books - probably because the font on them is about twice as large as that on the Dropbox book. And I didn't mean to imply you were copying Apple; just that the new cover is as bland looking as those I referenced. (Apple even makes it worse by using different colors) I hope that you'll at least keep similar category books the same color, like green for Mac specific books, red for iOS/iDevice related books, blue for non-specific internet like Dropbox, iCloud, etc. and Black for Windows/PC specific books. ;)
As I said, I'm sorry you don't like the new cover design (based on a single example), but we can't please everyone, and we do like it (after being bothered by our previous design and going through a lengthy design phase to address our concerns).
We thought about using patterns for the colors - either by category, as you suggest, or by author - but we've decided against it because in the new site design, the catalog will have a view that relies on thumbnails, and we worry that whatever scheme we choose, there will be too much of one color and not enough of the rest. So we're playing color choice by ear (by eye?) for each title. For instance, Joe wanted purple initially for this book, but we both agreed that blue was better in the end, because Dropbox uses a predominantly blue color scheme and the purple interior would have seemed jarring.
Dennis, thanks for the feedback here. We have some options in the design to add more elements, but Joe wanted to go with a simple, elegant look on this one, so we followed his lead. But, you are correct that it is a huge departure from the usual look.
Very interesting reading about your production process. Thank you.
When trying to create an ePub/mobi recently I was really disappointed that InDesign was unable to wrap text around a graphic for ePub, while we managed to do it in Pages.
Then, in searching for a solution for porting a print magazine to digital (made in InDesign, nice layout and looks great in pdf but need to sell on Amazon and Ibookstore coz don't want it pirated) we came to the conclusion that the only solution (excluding reading online) was to create an app.
There are many companies willing to charge you an arm and a leg for this - some with ongoing monthly charges - but we have yet to find a good solution that still leaves the 2nd mortgage on the house intact.
There are various possibilities, including Prss (not ready yet), MAZ, but I'm sure there must be others.
Does anyone have any ideas?
Maybe it is a good topic for an upcoming article in Tidbits.
Thanks for any ideas.
I'm a little down on the specific apps because it feels like a lot of work to put into a solution that has a single point of failure. Some major magazines (MIT Technology Review, the Financial Times, if I remember right) have moved away from apps to HTML5, which is much more malleable.
I am not an expert on EPUB, but a quick search seems to indicate that you can do what you want in InDesign. From my friends David Blatner and Anne-Marie Concepcion:
http://indesignsecrets.com/forum/general-indesign-topics/whats-the-best-way-to-wrap-an-image-captions-for-epub-export
Finally, I do want to address the piracy concern. We've been publishing Take Control for nearly 10 years with no DRM, and we've seen no impact of copying. We do some obvious things, like tell people to treat our ebooks like they would a physical book, and put a price on the cover, and while some copying has happened, there's no evidence it has ever impacted sales at all.
Absolutely fascinating to know about the complexity of your production process, alternatives, etc.. Also very well written.
My family has some histories and memoirs written by various family members that I would like to be able to distribute to everyone as .mobi (we mostly have Kindles). I don't need sophisticated tracking like you do, but do need to enter text and photos with captions. Can you suggest a decent solution for generating a mobi?
There's no way to make a Mobipocket file directly that I know of - it's always part of a larger system like Leanpub or converted from another format. The app you want for the conversion is Kindle Previewer from Amazon. It will take input in a variety of formats and output a Mobipocket file. The conversion can range from near-perfect to pretty horrible, depending on what you start with, so it's a matter of trial and error. Download it here:
http://www.amazon.com/gp/feature.html?ie=UTF8&docId=1000765261
Thanks! I'll give that a try.
I would suggest that you first create an epub, and then use calibre to convert it to the new AZW3 format. Unless people have a first gen kindle, AZW3 is a much better choice. Conversions from a good epub to AZW3 in Calibre have been very good in my (albeit limited) experience.
Alas, I've never found Calibre to create even acceptable output as soon as you get past plain text with no formatting. Kindle Previewer is pretty good most of the time.
We've just learned of another book publishing site: Lacuna Books.
https://www.lacunabooks.com/
It looks more akin to Booktype, PressBooks, and Inkling, although it sounds as though you wouldn't be able to change the design of your books at all. And you'd have to write in an online editor.
Thanks for sharing. Absolutely fascinating to know about the complexity of your production process, alternatives, etc.. Also very well written. I going to be fan of your blog.
I am a little bit confused about the issue with the PDFs, if they are using LaTeX to generate the PDFs then you should have excellent and complete control over the appearance of that PDF. this is what LaTeX is built for, including line lengths, fonts, headings, footnotes, links, &c.
The entire reason that anyone uses LaTeX is for its ability to produce gorgeous output and give you complete control.
It's been a few years since I messed with LaTeX, but when I used it to set papers for my wife's masters program she got comments several times from professors about how good her PDFs looked, and I'm no expert.
The problem is not LaTeX itself, but the extent to which it's customizable. We have a strong idea of what we want our ebooks to look like, and on the EPUB and Mobipocket side, we can control that completely with a custom.css file. But with the PDF, there are only a few settings that Leanpub exposes, so there's much less control.
This isn't a huge problem for us since Nisus Writer Pro makes pretty good PDFs on its own, and we can then convert to Markdown to create the EPUB and Mobi.