Scala Visual Studio Code

The following table shows the status of various features.

Open Visual Studio Code command windows by pressing CTRL+SHIFT+P and run Remote-Containers: Open Folder in Container command. This will initiate build and run of the development Docker container. Running Scala Spark job in Dev Container. Now everything is setup.

I am used to Visual Studio Code and was very happy to find this metals extension. I was able to 'import build' and fix issues like bumping up scala version in my projects but I am not sure how to reproduce this step to set up a run configuration and actually launch our app in Scala IDE. Visual Studio Code uses.vscode/launch.json to store user defined configurations, which can be run using: The Run - Start Debugging menu item or workbench.action.debug.start shortcut. The Run - Run Without Debugging menu item or workbench.action.debug.run shortcut. IDE support for Scala 3 is available in IDEs based on Scala Metals (e.g., Visual Studio Code, vim) and in IntelliJ IDEA. Using Visual Studio Code. To use Visual Studio Code on a Scala 3 project, ensure you have the Metals plugin installed. Then open the project directory in VS code.

Feature	Status	Notes
Import build	✅	Works with sbt, Gradle, Maven, Mill and Bloop.
Compile errors	✅	Syntax errors as you type and type errors on file save.
Completions	✅
Hover (type at point)	✅
Goto definition	✅	Works for project sources and Java/Scala library dependencies.
Document symbols	✅
Formatting	✅	Uses Scalafmt.
Find references	✅
Find implementations	✅
Workspace symbol	✅	Searches workspace sources and library dependencies. All-lowercase queries are case-insensitive.
Highlight	✅
Folding	✅
Rename symbol	✅
Run/Debug	✅
Code actions	✅
Organize imports	✅
Show implicits	✅

Requirements

Java 8 or 11 provided by OpenJDK or Oracle. Eclipse OpenJ9 is not supported,please make sure the JAVA_HOME environment variable points to a valid Java 8or 11 installation.

macOS, Linux or Windows. Metals is developed on macOS and every PR is testedon Ubuntu+Windows.

Scala 2.13, 2.12, 2.11 and Scala 3. Metals supports these Scala versions2.13.5, 2.12.13, 2.12.12, 2.12.11, 2.12.10, 2.13.3, 2.13.4, 2.11.12, 2.12.8,2.12.9, 2.13.0, 2.13.1, 2.13.2, 3.0.0-RC3, 3.0.0-RC2, 3.0.0-RC1, 3.0.0-M3,3.0.0-M2 and 3.0.0-M1. Note that 2.11.x support is deprecated and it will beremoved in future releases. It's recommended to upgrade to Scala 2.12 or Scala2.13

Installation

Install the Metals extension from theMarketplace.

Make sure to disable the extensionsScala Language ServerandScala (sbt)if they are installed. TheDotty Language Serverdoes not need to be disabled because the Metals and Dotty extensions don'tconflict with each other. However, if you want to work on Scala 3 code in aworkspace that was previously opened with Dotty Language Server you need tofirst remove .dotty-ide-artifact before opening the workspace with Metals.

Next, open a directory containing your Scala code. The extension activates whenthe main directory contains build.sbt or build.sc file, a Scala file isopened, which includes *.sbt, *.scala and *.sc file, or a standard Scaladirectory structure src/main/scala is detected.

Importing a build

The first time you open Metals in a new workspace it prompts you to import thebuild. Click 'Import build' to start the installation step.

'Not now' disables this prompt for 2 minutes.
'Don't show again' disables this prompt forever, use rm -rf .metals/ tore-enable the prompt.
Use tail -f .metals/metals.log to watch the build import progress.
Behind the scenes, Metals uses Bloopto import sbt builds, but you don't need Bloop installed on your machine torun this step.

Once the import step completes, compilation starts for your open *.scalafiles.

Once the sources have compiled successfully, you can navigate the codebase withgoto definition.

Custom sbt launcher

By default, Metals runs an embedded sbt-launch.jar launcher that respects.sbtopts and .jvmopts. However, the environment variables SBT_OPTS andJAVA_OPTS are not respected.

Update the 'Sbt Script' setting to use a custom sbt script instead of thedefault Metals launcher if you need further customizations like readingenvironment variables.

Speeding up import

The 'Import build' step can take a long time, especially the first time you runit in a new build. The exact time depends on the complexity of the build and iflibrary dependencies need to be downloaded. For example, this step can takeeverything from 10 seconds in small cached builds up to 10-15 minutes in largeuncached builds.

Consult theBloop documentationto learn how to speed up build import.

Importing changes

When you change build.sbt or sources under project/, you will be prompted tore-import the build.

Manually trigger build import

To manually trigger a build import, execute the 'Import build' command throughthe command palette (Cmd + Shift + P).

Run doctor

Execute the 'Run Doctor' through the command palette to troubleshoot potentialconfiguration problems in your workspace.

Configure Java version

The VS Code plugin uses by default the JAVA_HOME environment variable (vialocate-java-home) to locatethe java executable. To override the default Java home location, update the'Java Home' variable in the settings menu.

If this setting is defined, the VS Code plugin uses the custom path instead ofthe JAVA_HOME environment variable.

macOS

To globally configure $JAVA_HOME for all GUI applications, seethis Stackoverflow answer.

If you prefer to manually configure Java home through VS Code, run the followingcommand to copy the Java 8 home path.

Custom artifact repositories (Maven or Ivy resolvers)

Use the 'Custom Repositories' setting for the Metals VS Code extension to tellCoursier to try to download Metalsartifacts from your private artifact repository.

Use .jvmopts to set sbt options(https://www.scala-sbt.org/1.0/docs/Proxy-Repositories.html) forsbt bloopInstall which resolves library dependencies. You can also provide acustom sbt script (see 'Custom sbt launcher').

HTTP proxy

Metals uses Coursier to downloadartifacts from Maven Central. To use Metals behind an HTTP proxy, configure thesystem properties -Dhttps.proxyHost=… -Dhttps.proxyPort=… in one of thefollowing locations:

.jvmopts file in the workspace directory.
JAVA_OPTS environment variable, make sure to start code from your terminalwhen using this option since environment variables don't always propagatecorrectly when opening VS Code as a GUI application outside a terminal.
'Server Properties' setting for the Metals VS Code extension, which can beconfigured per-workspace or per-user.

Using latest Metals SNAPSHOT

Update the 'Server Version' setting to try out the latest pending Metalsfeatures.

Seehttps://scalameta.org/metals/docs/editors/vscode.html#using-latest-metals-snapshotto find the latest SNAPSHOT version.

Run the 'Reload Window' command after updating the setting for the new versionto take effect.

Files and Directories to include in your Gitignore

The Metals server places logs and other files in the .metals directory. TheBloop compile server places logs and compilation artifacts in the .bloopdirectory. The Bloop plugin that generates Bloop configuration is added in themetals.sbt file, which is added at project/metals.sbt as well as furtherproject directories depending on how deep *.sbt files need to be supported.To support each *.sbt file Metals needs to create an additional file at./project/project/metals.sbt relative to the sbt file. Working with Ammonitescripts will place compiled scripts into the .ammonite directory. It'srecommended to exclude these directories and files from version control systemslike git.

Show document symbols

Run the 'Explorer: Focus on Outline View' command to open the symbol outline forthe current file in the sidebar.

Run the 'Open Symbol in File' command to search for a symbol in the current filewithout opening the sidebar.

As you type, the symbol outline is also visible at the top of the file.

Go to parent code lenses

Metals has the ability to display code lenses that, when invoked, will go to theparent class that contains the definition of the method or symbol.Unfortunately, it might cause some lag in larger code bases, which is why it isnot enabled currently by default.

To enable the feature you need to modify the settingmetals.superMethodLensesEnabled to true.

Even without using the code lenses it's still possible to navigate the methodhierarchy using two commands:

Metals: Go to super method - immediately goes to the parent of the methodthe cursor is pointing to
Metals: Reveal super method hierachy - displays the full method hierachy andenables to move to any parent, it is best used with the Metals Quick Pickextension.

You can also bind those commands to a shortcut.

Create new project from template

It is possible using Metals to easily setup a new project using the exitinggiter8templates. This is an equivalent to the sbt new command, which uses the samemechanism. There is a great number of templates already available and it shouldbe easy to find something for yourself. To start the setup you can use theMetals: New Scala project command, which works as following:

Choose the template and then:
1. Use the proposed templates.
2. Choose 'Discover more' and then choose from the list downloaded from theGiter8 wiki page.
3. Input a custom Github repository following the organization/repo schema.
Navigate to the parent directory that you want to create your new project in.
Choose the name or accept the default one.
Choose whether to open a new window for the created project or use theexisting one.

The same command will be invoked when clicking the 'New Scala Project' button inthe Metals view.

If you feel like a template should be included in the default displayed ones donot hesitate to create aPRor file an issue.

Running and debugging your code

Metals supports running and debugging tests and main methods via theDebug Adapter Protocol.The protocol is used to communicate between the editor and debugger, which meansthat applications can be run the same as for any other language in the nativelysupported Run view. When using Metals the debugger itself isBloop, which is also responsible forstarting the actual process.

Users can begin the debugging session in two ways:

via code lenses

For each main or test class Metals shows two code lenses run | debug ortest | test debug, which show up above the definition as a kind of virtualtext. Clicking run or test will start running the main class or test withoutstopping at any breakpoints, while clicking debug or test debug will pauseonce any of them are hit. It's not possible to add any arguments or javaproperties when running using this method.

via a `launch.json` configuration

Visual Studio Code uses .vscode/launch.json to store user definedconfigurations, which can be run using:

The Run -> Start Debugging menu item or workbench.action.debug.startshortcut.
The Run -> Run Without Debugging menu item or workbench.action.debug.runshortcut.

If a user doesn't have anything yet saved, a configuration wizard will pop up toguide them. In the end users should end up with something like this:

You can also add an optional build target name, which is needed in case thereare more than one class with the same name or when launching a class fromoutside the project. Inside 'configurations': add the key buildTarget withyour target name, e.g. root:

The build target name corresponds to your project name. For example in sbt forlazy val interfaces = project the name of the build target will beinterfaces for sources and interfaces-test for tests. To make sure you havethe correct target names please run the command Metals: Run Doctor.

Multiple configurations can be stored in that file and can be chosen eithermanually in the Run view or can be picked by invoking a shortcut defined underworkbench.action.debug.selectandstart.

On type formatting for multiline string formatting

To properly support adding | in multiline strings we are using theonTypeFormatting method. The functionality is enabled by default, but you candisable/enable onTypeFormatting inside Visual Studio Code settings by checkingEditor: Format On Type:

Formatting on paste for multiline strings

Whenever text is paste into a multiline string with | it will be properlyformatted by Metals:

This feature is enabled by default. If you need to disable/enable formatting onpaste in Visual Studio Code you can check the Editor: Format On Paste setting:

Worksheets

Worksheets are a great way to explore an api, try out an idea, or code up anexample and quickly see the evaluated expression or result. Behind the scenesworksheets are powered by the great work done inmdoc.

Getting started with Worksheets

To get started with a worksheet you can either use the metals.new-scala-filecommand and select Worksheet or create a file called *.worksheet.sc. Thisformat is important since this is what tells Metals that it's meant to betreated as a worksheet and not just a Scala script. Where you create the scriptalso matters. If you'd like to use classes and values from your project, youneed to make sure the worksheet is created inside of your src directory. Youcan still create a worksheet in other places, but you will only have access tothe standard library and your dependencies.

Evaluations

After saving you'll see the result of the expression as a decoration at the endof the line. You may not see the full result for example if it's too long, soyou are also able to hover on the decoration to expand the decoration.

Keep in mind that you don't need to wrap your code in an object. In worksheetseverything can be evaluated at the top level.

Using dependencies in worksheets

You are able to include an external dependency in your worksheet by including itin one of the following two ways.

:: is the same as %% in sbt, which will append the current Scala binaryversion to the artifact name.

You can also import scalac options in a special $scalac import like below:

Coming from IntelliJ

Install theIntelliJ IDEA Keybindingsextension to use default IntelliJ shortcuts with VS Code.

IntelliJ	VS Code
Go to class	Go to symbol in workspace
Parameter info	Trigger parameter hints
Basic completion	Trigger suggest
Type info	Show hover
Expand	Fold
Extend Selection	Expand selection

-->

Learn how to use Apache Spark & Hive Tools for Visual Studio Code. Use the tools to create and submit Apache Hive batch jobs, interactive Hive queries, and PySpark scripts for Apache Spark. First we'll describe how to install Spark & Hive Tools in Visual Studio Code. Then we'll walk through how to submit jobs to Spark & Hive Tools.

Spark & Hive Tools can be installed on platforms that are supported by Visual Studio Code. Note the following prerequisites for different platforms.

Prerequisites

The following items are required for completing the steps in this article:

A serverless Apache Spark pool. To create a serverless Apache Spark pool, see Create Apache Spark pool using Azure portal.
Visual Studio Code.
Mono. Mono is required only for Linux and macOS.
A PySpark interactive environment for Visual Studio Code.
A local directory. This article uses C:HDSynaseexample.

Install Spark & Hive Tools

After you meet the prerequisites, you can install Spark & Hive Tools for Visual Studio Code by following these steps:

Open Visual Studio Code.
From the menu bar, navigate to View > Extensions.
In the search box, enter Spark & Hive.
Select Spark & Hive Tools from the search results, and then select Install:
Select Reload when necessary.

Open a work folder

To open a work folder and to create a file in Visual Studio Code, follow these steps:

From the menu bar, navigate to File > Open Folder.. > C:HDSynaseexample, and then select the Select Folder button. The folder appears in the Explorer view on the left.
In Explorer view, select the Synaseexample folder, and then select the New File icon next to the work folder:
Name the new file by using the .py (Spark script) file extension. This example uses HelloWorld.py.

Connect to your Spark pools

Sign in to your Azure subscription

Follow these steps to connect to Azure:

From the menu bar, navigate to View > Command Palette.., and enter Azure: Sign In:
Follow the sign-in instructions to sign in to Azure. After you're connected, your Azure account name shows on the status bar at the bottom of the Visual Studio Code window.

Set the default Spark pool

Reopen the Synaseexample folder that was discussed earlier, if closed.
Select the HelloWorld.py file that was created earlier. It opens in the script editor.
Right-click the script editor, and then select Synapse: Set default Spark pool.
Connect to your Azure account if you haven't yet done so.
Select a Spark pool as the default Spark pool for the current script file. The tools automatically update the .VSCodesettings.json configuration file:

Submit interactive Synapse PySpark queries to Spark pool

Users can perform Synapse PySpark interactive on Spark pool in the following ways:

Using the Synapse PySpark interactive command in PY file

Using the PySpark interactive command to submit the queries, follow these steps:

Reopen the Synaseexample folder that was discussed earlier, if closed.
Create a new HelloWorld.py file, following the earlier steps.
Copy and paste the following code into the script file:

The prompt to install PySpark/Synapse Pyspark kernel is displayed in the lower right corner of the window. You can click on Install button to proceed for the PySpark/Synapse Pyspark installations; or click on Skip button to skip this step.
If you need to install it later, you can navigate to File > Preference > Settings, then uncheck Hdinsight: Enable Skip Pyspark Installation in the settings.
If the installation is successful in step 4, the 'PySpark/Synapse Pyspark installed successfully' message box is displayed in the lower right corner of the window. Click on Reload button to reload the window.
From the menu bar, navigate to View > Command Palette.. or use the Shift + Ctrl + P keyboard shortcut, and enter Python: Select Interpreter to start Jupyter Server.
Select the python option below.
From the menu bar, navigate to View > Command Palette.. or use the Shift + Ctrl + P keyboard shortcut, and enter Developer: Reload Window.
Connect to your Azure account if you haven't yet done so.
Select all the code, right-click the script editor, and select Synapse: Pyspark Interactive to submit the query.
Select the Spark pool, if you haven't specified a default Spark pool. After a few moments, the Python Interactive results appear in a new tab. Click on PySpark to switch the kernel to Synapse PySpark, then, submit the selected code again, and the code will run successfully. The tools also let you submit a block of code instead of the whole script file by using the context menu:

Perform interactive query in PY file using a #%% comment

Add #%% before the code to get notebook experience.
Click on Run Cell. After a few moments, the Python Interactive results appear in a new tab. Click on PySpark to switch the kernel to Synapse PySpark, then, click on Run Cell again, and the code will run successfully.

Leverage IPYNB support from Python extension

You can create a Jupyter Notebook by command from the Command Palette or by creating a new .ipynb file in your workspace. For more information, see Working with Jupyter Notebooks in Visual Studio Code
Click on Run cell button, follow the prompts to Set the default spark pool (strongly encourage to set default cluster/pool every time before opening a notebook) and then, Reload window.
Click on PySpark to switch kernel to Synapse Pyspark, and then click on Run Cell, after a while, the result will be displayed.

Note

Ms-python >=2020.5.78807 version is not supported on this extention is a known issue.
Switch to Synapse Pyspark kernel, disabling auto-settings in Azure Portal is encouraged. Otherwise it may take a long while to wake up the cluster and set synapse kernel for the first time use.

Submit PySpark batch job to Spark pool

Reopen the Synaseexample folder that you discussed earlier, if closed.
Create a new BatchFile.py file by following the earlier steps.
Copy and paste the following code into the script file:
Connect to your Azure account if you haven't yet done so.
Right-click the script editor, and then select Synapse: PySpark Batch. Lizard for mac.
Select a Spark pool to submit your PySpark job to:

After you submit a batch job to spark pool, submission logs appear in the OUTPUT window in Visual Studio Code. The Spark UI URL and Spark Job Application UI URL are also shown. You can open the URL in a web browser to track the job status.

Access and manage Synapse Workspace

You can perform different operations in Azure Explorer within Spark & Hive tools for VSCode. From the Azure Explorer.

Launch workspace

From Azure Explorer, navigate to SYNAPSE, expand it, and display the Synapse Subscription list.
Click on Subscription of Synapse workspace, expand it, and display the workspace list.
Right-click a workspace, then select View Apache Spark applications, the Apache Spark application page in the Synapse Studio website will be opened.
Expand a workspace, Default Storage and Spark Pools are displayed.
Right-click on Default Storage, the Copy Full Path and Open in Synapse Studio are displayed.
- Click on Copy Full Path, the Primary ADLS Gen2 account URL will be copied, you can paste it where you need。
- Click on Open in Synapse Studio, the Primary Storage Account will be opened in Synapse Studio.
Expand the Default Storage, the Primary Storage Account is displayed.
Expand the Spark Pools, all spark pools in the workspace are displayed.

Known issues

ms-python >=2020.5.78807 version is not supported on this extention

Visual Studio Scala Plugin

'Failed to connect to Jupyter notebook.' is a known issue for python version >=2020.5.78807. It is recommended that users use the 2020.4.76186 version of ms-python to avoid this issue.