Brainstorming by Victor Semionov is licensed under CC BY 2.0
Apple to Deprecate Scripting Languages in Future Versions of macOS
When Apple announces major new versions of macOS, the company often includes details not just about the next release but also about what will come further in the future. The first macOS 10.15 Beta Release Notes has a Deprecations section that says:
Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for compatibility with legacy software. Future versions of macOS won’t include scripting language runtimes by default, and might require you to install additional packages. If your software depends on scripting languages, it’s recommended that you bundle the runtime within the app.
What does this note mean?
The most absolute interpretation is to assume that Apple means “[all] future versions” after Catalina won’t include the runtimes necessary to execute scripts written in these scripting languages.
However, Apple has at times deprecated longstanding components without removing them entirely. For instance, when Apple introduced launchd in Mac OS X 10.4 Tiger, it deprecated the venerable cron scheduling system; ten macOS versions later and cron is still humming along, available for users.
So this may all be a tempest in a teapot if Apple doesn’t end up following through on this threat. But that won’t become clear for at least a year, and in the meantime, we’ll all have to ponder the implications if Apple does kick these languages to the curb.
Why remove these scripting languages at all?
Three reasons: security, efficiency, and focus.
These languages are common in Unix-based systems and have been in Mac OS X since 2001, so why remove them now? Apple has progressively changed macOS to make it more secure, and that is likely the company’s primary motivation for this change. Gatekeeper prevents the automatic running of applications from unknown developers, but scripts are just text files that can do a great deal when executed by the scripting language runtime.
Removing scripting languages will also make macOS use less disk space. The runtime programs for each are small, but like all programming languages, they have many code libraries, common tasks script writers can call upon so they don’t have to reinvent a particular wheel. macOS comes with thousands of these library files that total over 150 MB. That’s not a lot in the scheme of things, but it’s also not trivial with an operating system that has to be downloaded millions of times.
Finally, there’s a principle that in business, you should focus on your strengths. Maintaining these languages has not been one of Apple’s; the company has not kept current with updates to the languages, and users can’t easily use standard tools to find and install additional libraries from the vast repositories available.
Who is affected, and what are their options?
How troubling the loss of these scripting languages will be depends on who you are.
Script Writers
The people who know these scripting languages the best will be the least affected. Those who write scripts for Web development, scientific analysis, or to get other work done in their preferred language already know that maintaining the default installations is not one of Apple’s strengths.
For a long time now, serious users of these languages have been installing separate copies in other folders so they can keep them up to date and easily install libraries from the repositories. That will almost certainly remain possible even after Apple removes the default installations.
App Developers
Mac apps are written in compiled languages but sometimes contain scripts within their app bundles to perform specific tasks. Apple’s deprecation notice recommends that developers include the script’s runtime executable within the application; this shifts responsibility to the developers.
Although multiple apps needing their own copies of Perl, for instance, might seem inefficient, developers need to include only the runtime and exact libraries they use, not a full scripting language installation. Some apps have already taken this route to keep using Java, and many modern apps rely on the Electron framework, which includes Node to run JavaScript.
Alternatively, developers can switch to using compiled code, either rewriting scripts from scratch in a different language or using a tool designed to convert a script into a binary executable format.
Mac Administrators
People who manage many computers for a school or business rely on automation, and automation, in turn, makes heavy use of scripting. While most app developers are already accustomed to dealing with Apple’s security measures for compiled code, Mac administrators are not, so it will be much harder for them to kick the scripting habit.
While some admins are fluent in one or more of these scripting languages, many more are like me, knowing just enough to get by and use scripts shared by the community. For instance, Greg Neagle’s Munki is a popular open-source tool for managing Mac software installs written in Python, but you don’t have to be fluent in Python to use it. Admins currently tend to accumulate useful scripts written in a variety of languages, but installing and maintaining multiple languages on every Mac is too much work, so in the future, admins are likely to choose just one, limiting the number of usable tools. For simpler tasks, admins can also switch to using shell scripts, which will be around at least as long as macOS includes the Terminal app.
Apple
Before removing these scripting languages from macOS, Apple will need to address its own reliance on them. Xcode includes many libraries in all three of the mentioned languages, but it should be trivial for Apple to add the runtimes to Xcode’s already enormous install. iMovie includes a lone Perl script, and it wouldn’t surprise me if Apple’s pro apps like Final Cut Pro X and Logic Pro X also contain some.
In addition to Apple’s apps, macOS 10.14 Mojave contains over 175 scripts outside the folders devoted to Perl, Python, and Ruby. Some are a part of the language, just in a different place, but the rest serve varied purposes. Once I realized how many there were, it seemed much less likely that all three languages would be removed in the macOS version after Catalina.
Everyone Else
It’s hard to say how many everyday users rely on at least one app that contains at least one script. For actively maintained apps, the developer will solve this for the end user. For some open-source projects, particularly cross-platform ones for which macOS is not the primary platform, project contributors may not get around to incorporating a solution.
For those cases and for apps that are no longer maintained, there is a solution: install the scripting language yourself. Each scripting language has its own method of Mac installation, but the simplest and most common method is a general package management system like Homebrew. Homebrew itself is made up of Ruby scripts; I expect its developers will heed Apple’s warning and include that runtime before Ruby disappears in a future version of macOS.
If you’re curious how many of your applications contain scripts in these languages, try copying this command and pasting it into Terminal:
find /Applications -type f | while read in ; do if file -b "${in}" | grep -q 'Perl\|Python\|Ruby' ; then echo "${in}" ; fi ; done
The first parameter after find is the folder to be checked. On my Mac, Plex Media Server contains many Python scripts and BBEdit includes a couple, as do a handful of other applications, generally from smaller companies or open-source projects.
Start Planning
The point of deprecation notices is to give people time to make changes, but typically these notices don’t operate on a fixed schedule. If Apple itself can’t prioritize ending its reliance on scripts or if the company hears from significant constituencies that they’re not ready, one or more of the languages may stick around.
Even if Apple removes all the scripting languages from the macOS 10.16 release in 2020, it won’t be as significant a change as other deprecations, such as the end of 32-bit app support coming to fruition in macOS Catalina. Unlike that change, if Apple removes a scripting language, you can always put it back.
One final note. Swift can be used as a scripting language, and although its runtime isn’t currently included with macOS, it’s possible that Apple will anoint it as the Mac’s scripting language of choice at some point in the future.
Curtis Wilcox is an Apple-focused IT administrator at Harvard University, where he supports classroom instruction, media production, and projects aimed at improving accessibility.
Brett Terpstra has penned a blog post about this as well:
Out of curiosity, I ran the Terminal code you provided in the article to
see what I use that might be affected. (Warning to anyone else who runs
it: it can take a long time if you have more than a handful of apps
installed. It took more than a half hour just to find all the affected
files in Xcode.app for me.)
Aside from the things you already mentioned that you found affected, one
thing stuck out to me like a sore thumb: anything that uses ExifTool
will be affected, because ExifTool appears to be more or less 100% Perl.
This includes metadata-management software like A Better Finder
Attributes and A Better Finder Rename, as well as a lot of
photo-management software, including the widely used Lemkesoft Graphic
Converter.
Mark D. McKean
[email protected]
I feel certain that won’t be the case, at least not in the near term. The only thing that changes is that the default for newly installed macOS 10.15 users (not upgrades) will be to use zsh in Terminal. I would also guess that Apple will be converting all their own scripts as rapidly as possible to use zsh and it is clear that the current version of bash, already an ancient version 3.2 (current being 5.1 with 7.0 about ready for release), will never be updated by Apple, since it’s currently licensed under GPLv3.
Homebrew’s current installation method requires Ruby to already be present on a Mac but I don’t think it will be difficult to change the installation method to not require Ruby. From there, Homebrew can include the Rudy runtime so all its formulas don’t have to be rewritten.
Brett’s collection of projects is an example of the “vast corpus of code” I referred to when I originally posted this news to the list. They’re basically little programs for people to use without knowledge of the scripting languages themselves, at most you need to know the fundamentals of running code in Terminal. Even within Markdown Service Tools, one project, he has used more than one scripting language within its Automator workflows. I would be surprised if you can embed a runtime within a workflow so even if he ports all the scripts to a single language, first installing that scripting language will be a system requirement for their use. It’s potentially a lot of work for him with no apparent benefit to him or the users of his code.
Brett wrote:
I don’t think that follows at all. Actual shell scripts will still be present so the action will still be useful in Automator without installing one of these scripting languages. Whether Automator will last is another matter, I have no sense of how much it’s used.
Armin Briegel has started an eight (!) part series on Moving to zsh.
Regarding the last bit, about using Swift as a scripting language, the runtime isn’t present but there is a “stub” program so if you try to use it, you get a dialog:
When the deprecation notice says “Future versions of macOS won’t include scripting language runtimes by default,” I wonder if they’ll do something like this, prompt you to install the scripting language if you try to use it. I’d rather they not all be rolled into the command line developer tools which already includes the ability to compile a number of different non-scripting languages.
Excellent point. Couldn’t agree more. Of all the bloat Apple packs into a macOS install, old interpreters that allow us to easily use all kinds of scripts in a lowest common denominator fashion (exactly as the above quote so eloquently summarizes) is about the last thing I’d remove. Sure, people can install these on their own, and those that need the very latest version will do that and they’ll either already know how or learn it (in my case for example for stuff like bash or gcc or make), but that doesn’t mean you have to force everybody into that corner. In fact if you do, some of them will jump ship to a world where that stuff is taken care of from the start.
One of the key OS X strengths was that it came with a whole lot of good UNIX stuff courtesy of its NeXT heritage (terminal, scripting languages, compilers, X11, etc.). That’s one of the key components in making it so attractive to science. This latest Apple move is just one more chipping away at exactly that pillar. And don’t anybody give me the “well scientists can install that stuff on their own”. In the real world it just doesn’t work like that. People hate futzing around with knock off package managers or having to make compile stuff from source. If I have to install all those completely standard *NIX tools from scratch, and I then first have to install all the necessary nuts and bolts just to be able to start that actual install itself, … well shoot me already. I might as well go for a cheaper PC with CentOS right from the start. It’s easier. Ironically, it was Apple that used to be all about ease of use.
I understand these tools may no longer be installed as a default, but what if you install the developer’s kit? Many developers fell in love with the Mac because it was a true Unix machine with all of the Unix tools, standard C libraries, and APIs. Windows didn’t.
You go to companies and see tech people with company provided Windows systems gathering dust. Instead, they bring in their MacBooks and use them. Many people became Apple fans because of this. At least, install these languages if I install Xcode.
As for zsh, what’s the technical term? It sucks. Zsh is bloated and has a tendency to crash. The idea is to combine all possible syntaxes into a single shell. Hate your life and want to suffer? Write a script in csh syntax! This bloats the size of zsh. And in the end, many of the commands and functions operate differently from bash.
It’s not entirely Apple’s fault. Apple for legal reasons cannot use software under the GNUv3 license which includes all newer versions of the Bash shell. The latest version of bash on the Mac is versions 3 abs bash is now on version 5.
A lot of software license under a dual license of GNU and under a version of an Apache or MIT license. This allows companies like Apple to license the software under the alternative license. Bash doesn’t.
I do not think we will know whether Apple will continue to include them with Xcode and / or Command Line Tools when the time comes to remove them from a standard macOS installation, so we will all have to wait and see.
Even if they don’t, I feel certain that services like MacPorts and Homebrew will continue to make them available for users that have need for them.
Re:
Hardly. Apple is completely free to use and ship GPLv3 (GNUv3 is a typo) software, as do many other big companies including Microsoft (Microsoft is adding the Linux command line to Windows 10 - The Verge). As the article on thenextweb which you link to demonstrates, it is Apple’s choices that cause them problems, and they are actually choices which curtail the freedoms and options of users as well as developers, both directly and indirectly as we see with this new decision.
I (and evidently Microsoft) don’t see why the patent clause of GPLv3 is any sort of issue for Apple unless they modify Bash itself with some functionality upon which they want to claim and enforce patents.
This is yet another link in the chain they impose, like back in 1994 when Apple tried to prevent the world from using common sense GUI “look and feel” techniques which they themselves lifted from Xerox PARC.
Linux itself is published under GPLv2 and not GPLv3 which Is a big difference. GPLv2 requires any changes you make in a project to be submitted back into the project, but that’s it. GLPv3 can require you to open your code and IP. Apple has no problem with version 2 of the license and used GNU BASH up until that project came with version 3 of the license.
Microsoft’s Linux Subsystem actually uses no Linux kernel code itself. It is a virtual Hyper-V system that creates a ext-4 formatted disk. You can download any Linux distro you want. This is allowed under GPLv2.
macOS itself is Unix and is licensed under the BSD license. Most of the command line tools are not GNU versions of those tools. This has caused some consternation when developers use some GNU extensions of these tools and discover they don’t work on macOS’ version of the tool.
GPLv3 has been a major problem for many companies. Use of the license can force you to open source much of your code, or allow users to modify your code which would be a support nightmare. At my last company, we depended heavily upon open source and had to be very careful of all the licenses used. This included various JavaScript libraries, various startup scripts, and other libraries that linked to A particular tool you planned to use.
Yes, GPLv3 is different than GPLv3, with more protections for users. But can you cite any specific examples of cases where distribution of a 3rd-party GPLv3 tool like Bash was found to “force you to open source much of your code”? I think that is a very mistaken notion. (Again, of course it forces you to open source any enhancements to Bash itself that you might choose to make, but that is quite a different thing.) The situation can of course be different with libraries you want to incorporate into your code (though those would normally be under LGPL which avoids most such issues), but even Intel and Oracle ship Linux distributions with loads of GPLv3 code like Bash. I’ve looked and found no specific example where something like Bash could cause Apple that kind of hassle.
Good grief! First Server, and now this.
One of the selling points of Apple for our school district was the fact that one machine could support the three major operating systems (macOS, Windows, Linux) and the icing on the cake was that macs had built-in unix scripting (albeit old versions). Back in the good old days I had a mac server that ran both an OpenDirectory environment for Macs and an Active Directory domain for PCs at one of our schools. And it did both jobs quite well.
Those days are gone, but at least I had scripting to look forward to…until now.
We use scripting heavily for mac deployment and remote management. Apple Remote Desktop has a very handy Unix option which lets you push and run scripts across multiple macs in parallel on the fly. I manage most of our initial mac setups with scripts (because our district wouldn’t buy mac licenses for JAMF Pro). Why, oh why Apple, do you want to take that away?
It seems like Apple is using “security” as an excuse to wrest more and more control away from the power user and admin types, so they can turn macs into toasters like iPads. And we’re supposed to be happy about it? What happened to being user-friendly?
The big brother in the old 1984 Macintosh commercial isn’t the voice of Microsoft any more. In 2020 it’s the voice of Apple…
I wasn’t talking about the Microsoft Linux Subsystem. Microsoft also ships distros complete with GPLv3 code, and even modify it enough that they charge for some versions: Microsoft Announces $20 Linux-Based Distro for Windows 10 1809.
That Linux distro that Microsoft provides is completely separate from Windows. It runs in the Windows subsystem. It would be perfectly okay if Apple created a virtual machine for running Linux and then distributed a Linux distro. All Apple would have to provide is the source for that distro (or tell you how you can download it for free).
The Mac is a Unix based system and the various Unix/GNU tools run directly under MacOS and not a virtual machine. If Apple supplied a GNU tool that used a GPLv2 license, and that GNU tool linked to a macOS library with proprietary code, that would be perfectly okay. This was the case with older versions of BASH.
However, if a tool is under the GPLv3 license and links to a macOS library, that macOS library’s source would have to be fully opened up. That’s the problem with newer versions of BASH. They are only licensed under GPLv3. If Apple provides a compiled versions of these tools, the source code for these tools and all libraries and resources they link to must also be open sourced.
Apple could get around this by supplying the BASH source and the needed compiling tools and configuration files and let the user compile it themselves. But, this defeats the purpose of being a Unix system. You can already do this via Homebrew is MacPorts.
I don’t think searching for such a specific example would be useful here. Apple has always made it clear that they will not deal with GPL v3 over several years now. Whether it has ever been enforced or not doesn’t appear to matter. The multitude of lawyers at Apple must have decided it’s not worth the risk.
-Al-
They’re shipping the entire Linux distro with Bash. They include all the source. There is absolutely no proprietary code. Even the operating system is open source.
MacOS isn’t open source and the idea is to have the Unix tools like cp, mv, rm, and shells like BASH run directly under macOS. There are various versions of many of these tools, and the Mac mainly uses the BSD version of these tools. This is because the BSD license is much less strict.
The GPLv3 license is a big problem for many companies. I linked to one company earlier that wanted to use zookeeper in their product. That’s an Apache licensed tool, but for some reason, the installation required the GNU version of a particular tool for configuration. If they offered that version of that GNU tool, they could end up open sourcing their whole project.
This is a major issue for commercial projects. That’s why they use tools like BlackDuck to make sure they’re in compliance. XimpleWare have made themselves known for suing commercial companies for license noncompliance.
Thanks for the alert, Curtis!
Do please provide a link to that “stub” program and any current information about using Swift as a scripting language. I gave up waiting in vain a few years ago, and have long been mourning the demise of Hamish Sanderson’s SwiftAutomation and its predecessor, appscript!
I’m talking about using Swift as a scripting language in the same way the deprecated languages are used. I’m not talking about using it to control macOS applications the way AppleScript does, that appears to be what SwiftAutomation and appscript are for. As far as I know, nothing has gotten better (or worse) on that front. It looks like the SwiftAutomation developer is still working on it.
The “stub” is /usr/bin/swift, it’s present going back at least to macOS Sierra. Entering ‘swift’ in Terminal is sufficient to trigger the GUI prompt to install the developer tools. When the command line developer tools (or full Xcode) are installed, I don’t think /usr/bin/swift is replaced, it simply is able to access the libraries it needs to do anything.
Writing in the Swift language is basically the same whether compiled or run as a script so you can learn it from the many resources for learning Swift, skip the stuff specific to iOS and GUIs. This short blog post covers the most basic aspects of scripting with it (the shebang, setting execute file permissions, input/output): Scripting and Compiling Swift on the Command Line.
Bit late to this thread… BUT, somehow I never heard anything about good old AppleScript. For the first time in MANY years I turned to it for something that might make my life just a tiny bit better… was building it in steps, got the first step working from the Script/Run, turned it into an app… hmm THAT would not run at all. Sierra. Did I miss it’s death?
Not sure what might have caused that, but AppleScript is definitely alive, well and fully supported in all current versions of macOS. There have been some updates along the way that caused a few older scripts to fail, but all of mine were easily fixed by simply opening it in Script Editor, recompiling and saving.
Also used this code in the terminal. Strange thing here that
findonly detects Python scripts, but Ruby and Perl files are simple ASCII text. I wonder if it’s just on my local machine or it’s a common thing.I think
{locate "*.py" && locate "*.rb" && locate "*.pl"} | grep [pattern]and then filter the results might be a better solution.None of the scripting languages listed have gone away but PHP has been removed from macOS Monterey.
macOS Big Sur actually included a small update to Perl and the OS includes more scripts written in Perl than Catalina did.