Xcode​Kit and Xcode Source Editor Extensions

When we last wrote about extending Xcode in 2014, we were living in a golden age, and didn’t even know it.

Back then, Xcode was a supposedly impenetrable castle that we’d leaned a couple of ladders against. Like a surprisingly considerate horde, we scaled the walls and got to work on some much-needed upkeep. Those were heady days of in-process code injection, an informally sanctioned and thriving ecosystem of third-party plugins — all backed up by an in-app package manager. For a while, Apple tolerated it all. But with the introduction of System Integrity Protection in 2016, the ladders were abruptly kicked away. (Pour one out for Alcatraz why don’t we, with a chaser for XcodeColors. Miss you buddy.)

Plugins allowed us to tweak pretty much everything about Xcode: window layout, syntactic and semantic highlighting, changing UI elements, boilerplate generation, project analysis, bindings for something called Vim (?). Looking back at NSHipster’s favorites, some are now thankfully included as a standard feature: inserting documentation comments, switch statement autocompletion or — astonishingly — line breaks in the issue navigator. Most of the inventive functionality that plugins added, though, has just plain gone.

Xcode 8 proposed a solution for the missing plugins in the form of Source Editor Extensions. Like other macOS extensions, they can be sold via the App Store or distributed independently. But some bad, if old, news: unlike plugins, these new extensions are seriously limited in scope. They allow pure text manipulation, instigated by the user from a menu command, on one source file at a time — none of the fun stuff, in other words.

Source Editor Extensions have remained unchanged since introduction. We’ll discuss signs that might point to interesting future developments. But if IDEs with an open attitude are more your thing, there’s not much to see here yet.

Let’s start, though, by looking at the official situation today:

Source Editor Extensions

By now, Apple platform developers will be familiar with extension architecture: separate binaries, sandboxed and running in their own process, but not distributable without a containing app.

Compared to using a tool like Homebrew, installation is undoubtedly a pain:

After finding, downloading and launching the containing app, the extension shows up in the Extensions pane of System Preferences. You can then activate it, restart Xcode and it should manifest itself as a menu item.
(App Store reviewers love this process.)

That’s the finished result. To understand how you get to that point, let’s create a simple extension of our own. This sample project transforms TODO, FIXME and MARK code marks to be uppercased with a trailing colon, so Xcode can recognize them and add them to the quick navigation bar. (It’s just one of the rules more fully implemented by the SwiftFormat extension.)

Creating a Source Editor Extension

Create a new Cocoa app as the containing app, and add a new target using the Xcode Source Editor Extension template.

The target contains ready-made XCSourceEditorExtension and XCSourceEditorCommand subclasses, with a configured property list.

Both of those superclasses are part of the XcodeKit framework (hence the XC prefix), which provides extensions the ability to modify the text and selections of a source file.

Display Names

User-facing strings for an extension are sprinkled around the extension’s Info.plist or defined at runtime:

Display text	Property	Definition
Extension name, as shown in System Preferences	Bundle Display Name	Info.plist
Top level menu item for extension	Bundle Name	Info.plist
Individual menu command	XCSourceEditorCommandName	Info.plist
	XCSourceEditorCommandDefinitionKey.nameKey	Runtime

Menu Items

The only way a user can interact with an extension is by selecting one of its menu items. These show up at the bottom of the Editor menu when viewing a source code file. Xcode’s one affordance to users is that keybindings can be assigned to extension commands, just as for other menu items.

Each command gets a stringly-typed identifier, display text, and a class to handle it, which are each defined in the extension target’s Info.plist. Alternatively, we can override these at runtime by providing a commandDefinitions property on the XCSourceEditorExtension subclass. The commands can all be funneled to a single XCSourceEditorCommand subclass or split up to be handled by multiple classes — whichever you prefer.

In our extension, we just define a single “Format Marks” menu item:

    var commandDefinitions: [[XCSourceEditorCommandDefinitionKey: Any]] {
        let namespace = Bundle(for: type(of: self)).bundleIdentifier!
        let marker = MarkerCommand.className()
        return [[.identifierKey: namespace + marker,
                 .classNameKey: marker,
                 .nameKey: NSLocalizedString("Format Marks",
                 comment: "format marks menu item")]]
    }

When the user chooses one of the menu commands defined by the extension, the handling class is called with perform(with:completionHandler:). The extension finally gets access to something useful, namely the contents of the current source code file.

Inputs and Outputs

The passed XCSourceEditorCommandInvocation argument holds a reference to the XCSourceTextBuffer, which gives us access to:

• completeBuffer, containing the entire text of the file as a single String
• another view on the same text, separated into lines of code
• an array of current selections in terms of lines and columns, supporting multiple cursors
• various indentation settings
• the type of source file

With text and selections in hand, we get to do the meaty work of the extension. Then XcodeKit provides two ways to write back to the same source file, by mutating either the completeBuffer or the more performant lines property. Mutating one changes the other, and Xcode applies those changes once the completion handler is called. Modifying the selections property updates the user’s selection in the same way.

In our example, we first loop over the lines of code. For each line, we use a regular expression to determine if it has a code mark that needs reformatting. If so, we note the index number and the replacement line. Finally we mutate the lines property to update the source file, and call the completion handler to signal that we’re done:

func perform(with invocation: XCSourceEditorCommandInvocation,
             completionHandler: @escaping (Error?) -> Void ) -> Void
{
    replaceLines(in: invocation.buffer.lines, by: formattingMarks)
    completionHandler(nil)
}

func replaceLines(in lines: NSMutableArray,
                  by replacing: @escaping (String) -> String?)
{
    guard let strings = lines as? [String] else {
        return
    }

    let newStrings: [(Int, String)] = strings.enumerated().compactMap {
        let (index, line) = $0
        guard let replacementLine = replacing(line) else {
            return nil
        }
        return (index, replacementLine)
    }

    newStrings.forEach {
        let (index, newString) = $0
        lines[index] = newString
    }
}

func formattingMarks(in string: String) -> String? {
  /* Regex magic transforms:
     "// fixme here be 🐉"
     to
     "// FIXME: here be 🐉"
  */
}

Development Tips

Debugging

Debugging the extension target launches it in a separate Xcode instance, with a dark status bar and icon:

Sometimes attaching to the debugger fails silently, and it’s a good idea to set a log or audible breakpoint to track this:

func extensionDidFinishLaunching() {
    os_log("Extension ready", type: .debug)
}

Extension Scheme Setup

Two suggestions from Daniel Jalkut to make life easier.
Firstly add Xcode as the default executable in the Extension scheme’s Run/Info pane:

Secondly, add a path to a file or project containing some good code to test against, in the Run/Arguments panel of the extension’s scheme, under Arguments Passed On Launch:

Testing XcodeKit

Make sure the test target knows how to find the XcodeKit framework, if you need to write tests against it. Add ${DEVELOPER_FRAMEWORKS_DIR} as both a Runpath and a Framework Search Path in Build Settings:

Using pluginkit

During development, Xcode can become confused as to which extensions it sees. It can be useful to get an overview of installed extensions using the pluginkit tool. This allows us to query the private PluginKit framework that manages all system extensions.

Here we’re matching by the NSExtensionPointIdentifier for Source Editor extensions:

$ pluginkit -m -p com.apple.dt.Xcode.extension.source-editor

+    com.apple.dt.XCDocumenter.XCDocumenterExtension(1.0)
+    com.apple.dt.XcodeBuiltInExtensions(10.2)
     com.Swiftify.Xcode.Extension(4.6.1)
+    com.charcoaldesign.SwiftFormat-for-Xcode.SourceEditorExtension(0.40.3)
!    com.hotbeverage.accesscontrolkitty.extension(1.0.1)

The leading flags in the output can give you some clues as to what might be happening:

• + seems to indicate a specifically enabled extension
• - indicates a disabled extension
• ! indicates some form of conflict

For extra verbose output that lists any duplicates:

$ pluginkit -m -p com.apple.dt.Xcode.extension.source-editor -A -D -vvv

If you spot an extension that might be causing an issue, you can try manually removing it:

$ pluginkit -r path/to/extension

Finally, when multiple copies of Xcode are on the same machine, extensions can stop working completely. In this case, Apple Developer Relations suggests re-registering your main copy of Xcode with Launch Services (it’s easiest to temporarily add lsregister’s location to PATH first):

$ PATH=/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support:"$PATH"
$ lsregister -f /Applications/Xcode.app

Features and Caveats

Transforming Source Code

Given how limited XcodeKit’s text API is, what sorts of things are people making? And can it entice tool creators away from the command line? (Hint: 😬)

• Linting and style extensions (reformatting based on rules, wrapping comments, alignment, whitespace adjustment, code organization, profanity removal)
• Coding helpers (insert and remove caveman debugging statements at function calls; moving import statements up to the top of the file from any location)
• Translators, from one language to another (JSON or Objective-C to Swift)
• Boilerplate generators (init statements, extraction of a protocol from a class, coding keys)
• Extracting code to put in a different context (a playground; a gist)

All the tools mentioned above are clearly transforming source code in various ways. They’ll need some information about the structure of that code to do useful work. Could they be using SourceKit directly? Well, where the extension is on the App Store, we know that they’re not. The extension must be sandboxed just to be loaded by Xcode, whereas calls to SourceKit needs to be un-sandboxed, which of course won’t fly in the App Store. We could distribute independently and use an un-sandboxed XPC service embedded in the extension. Or more likely, we can write our own single-purpose code to get the job done. The power of Xcode’s compiler is tantalizingly out of reach here. An opportunity, though, if writing a mini-parser sounds like fun (🙋🏼, and check out SwiftFormat’s beautiful lexer implementation for Swift).

Context-free Source Code

Once we have some way to analyze source code, how sophisticated an extension can we then write? Let’s remember that the current API gives us access to a file of text, but not any of its context within a project.

As an example, say we want to implement an extension that quickly modifies the access level of Swift code to make it part of a framework’s API. So an internal class’s internal properties and functions get changed to public, but private or fileprivate implementation details are left alone.

We can get most of the way there, lexing and parsing the file to figure out where to make appropriate changes, taking into account Swift’s rules about access inheritance. But what happens if one of these transformed methods turns out to have a parameter with an internal type? If that type is declared in a different file, there’s no way for our extension to know, and making the method public will cause a build error: “Method cannot be declared public because its parameter uses an internal type”.

In this example, we’re missing type declarations in other files. But complex refactorings can need information about how an entire codebase fits together. Metadata could also be useful, for example, what version of Swift the project uses, or a file path to save per-project configuration.

This is a frustrating trade-off for safety. While it’s feasible to transform the purely syntactic parts of isolated code, once any semantics come into play we quickly bump up against that missing context.

Output

You can only output transformed text back to the same source file using the extension API. If you were hoping to generate extensive boilerplate and insert project files automatically, this isn’t supported and would be fragile to manage via the containing app. Anonymous source file in/out sure is secure, but it isn’t powerful.

Heavyweight Architecture; Lightweight Results

Most extensions’ containing apps are hollow shells with installation instructions and some global preferences. Why? Well, a Cocoa app can do anything, but the extension doesn’t give us a lot to work with:

• As creators, we must deal with sandboxed communications to the containing app, the limited API and entitlements. Add complete sandboxing when distributing through the App Store.
• As users we contend with that convoluted installation experience, and managing preferences for each extension separately in the containing apps.

It’s all, effectively, for the privilege of a menu item. And the upshot is apparent from a prominent example in the Mac App Store, Swiftify: they suggest no fewer than four superior ways to access their service, over using their own native extension.

The Handwavy Bit

To further belabor the Xcode-as-castle metaphor, Apple has opened the gate just very slightly, but also positioned a large gentleman behind it, deterring all but the most innocuous of guests.

Extensions might have temporarily pacified the horde, but they are no panacea. After nearly three years without expanding the API, it’s no wonder that the App Store is not our destination of choice to augment Xcode. And Apple’s “best software for most” credo doesn’t mean they always get the IDE experience right cough image literals autocompletion cough, or make us optimistic that Xcode will become truly extensible in the style of VSCode.

But let’s swirl some tea leaves and see where Apple could take us if they so wished:

• Imagine a world where Xcode is using SwiftSyntax directly to represent the syntax of a file (a stated goal of the project). Let’s imagine that XcodeKit exposes Syntax nodes in some way through the extension API. We would be working with exactly the same representation as Xcode — no hand-written parsers needed. Tools are already being written against this library — it would be so neat to get them directly in Xcode.
• Let’s imagine we have specific read/write access to the current project directory and metadata. Perhaps this leverages the robust entitlements system, with approval through App Review. That sounds good to create extensive boilerplate.
• Let’s expand our vision: there’s a way to access fuller semantic information about our code, maybe driven via the LSP protocol. Given a better way to output changes too, we could use that information for complex, custom refactorings.
• Imagine invoking extensions automatically, for example as part of the build.
• Imagine API calls that add custom UI or Touch Bar items, according to context.
• Imagine a thriving, vibrant section of the Mac App Store for developer extensions.

---

Whew. That magic tea is strong stuff. In that world, extensions look a lot more fun, powerful, and worth the architectural hassles. Of course, this is rank speculation, and yet… The open-source projects Apple is committed to working on will — eventually — change the internal architecture of Xcode, and surely stranger things are happening.

For now, though, if any of this potential excites you, please write or tweet about it, submit enhancement requests, get involved on the relevant forums or contribute directly. We’re still hoping the Xcode team renders this article comprehensively obsolete, sooner rather than later 🤞.