Marketplace Extensions Vetting Process and Security Model

We get a lot of queries about the extension vetting process we have in Marketplace, as well as the security model for Azure DevOps extensions. Marketplace serves the extension ecosystem for Visual Studio Code, Visual Studio IDE and Azure DevOps; and we strive to make sure that the extensions available in the marketplace are of high quality and are secure. Here are some of the things we do in our pipeline to ensure safety:

  • We perform virus scan on both initial publish and subsequent update of an extension so that you aren’t getting malware from Marketplace. This also includes checking for npm and other dependent packages for known vulnerabilities like this.
  • We perform content scan on both initial publish and subsequent update of an extension so that you aren’t getting adult, offensive, CSAM (child sexual abuse material) and terrorist content from Marketplace.

Also, extensions can only operate within the scope and permissions it has been granted. For example: an extension which has only read permissions on work items cannot modify your features, bugs or any other type of work item. For Azure DevOps extensions, you can review the permissions that an extension requires in the Select Organization step as shown below,

extension permissions

For publishers:

  • Before a publisher can publicly list an Azure DevOps extension in Marketplace, we need to verify them. This requires that they send an email to us from their account and state their social presence (linkedin, twitter, github, organization etc.). This way we know them a little better and also get to understand that it is not a bot publishing an extension.

Apart from the above, we also rely on the community to bubble up quality extensions and report the ones that seem suspicious. We have Q&A and Ratings on each extension so that you can engage with an extension’s publisher(s) and have a meaningful dialogue. You can also use this Q&A capability to understand the software development life cycle practices the publisher follows, their test matrix and exit criteria. Do see what others are asking and if the publisher responds to them soon and in a meaningful way.

I also want to call out a few other important points,

  • There is no formal code review or evaluation process for extensions published to the Marketplace
  • Azure DevOps web extensions execute in the browser (iFrame) and can pull in scripts from other services (i.e. the extension is not limited to only the scripts it included within its package)
  • Azure DevOps web extensions run in isolation in the browser (a sandboxed iFrame) which prevents them from accessing Azure DevOps/TFS data or APIs they are not approved to access. You may have noticed that the admin is prompted to approve permissions when installing an extension that requires new permissions (e.g. “read only access to work items”). The extension is unable to do more than what it is authorized for at install time (and beyond what the current user of the extension is able to do). One way to protect yourself here is to carefully analyze the scopes being requested by the extension. If something doesn’t seem right, don’t install and contact vsmarketplace at microsoft dot com
  • There is no way for a web extension to install or run any code directly on the Azure DevOps or TFS servers. Build and Release tasks though, are a little different. These tasks get downloaded on the agent machine (which can be hosted or on-premise) and get executed. These tasks can download and invoke other code. To know more about the agent security model see this https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/pools-queues?view=azure-devops#security
  • Extensions should continue working after the sprintly Azure DevOps deployments and TFS upgrade since we maintain backward compatibility for all APIs that an extension can call. Any data stored by the extension (in a service provided by Azure DevOps and TFS) is not touched during these deployments/upgraded.

If you still have concerns with security, we recommend you install the extension on an isolated organization first to rest your concerns. This can be an internal test organization. After you’re satisfied with your testing, you can proceed to install the extension on your production organization. If you see compatibility issues after an update of the extension, please reach out to the publisher and report the same.

Hope all this information was helpful. Please reach out to vsmarketplace at microsoft dot com in case you have any more questions!

If you see a suspicious extension on Marketplace, do report it to us at vsmarketplace at microsoft dot com. You can also use the ‘Report Abuse’ link on the extensions details page to email us.

 

 

 

 

 

 

 

Ciao.

Ratings and reviews on VS Marketplace!

We’ve enabled a rating and review system on VS Marketplace for VSTS and VSCode extensions. Until now, download count of an extension served as a proxy for estimating the quality of an extension but no more!

You can see a 5 star rating on the extension on the marketplace homepage. Note that rating and review was already available for Visual Studio extensions. This enables it for VSTS and VS Code extensions as well.

Hovering over the stars shows you the exact rating and the number of people who have rated this extension.

image

Clicking on the extension takes you to the details page, where we show the average rating of the extension and the number of ratings on the banner,

image

If you notice carefully, you can see that that color of the stars on the banner will change between orang-ish or red-ish based on the background color on which it is rendered. This is done so that the stars have a nice contrast and can be seen clearly against the background. Here’s an example: red stars on a light background and orange stars on a dark background,

image

image

You can click on the stars to scroll down to the details section,

image

The detailed section consists of, as you can probably guess, details of the reviews. You need to be logged-in to leave a review, you can use your Microsoft Account or any other AAD backend account for this. The detailed section shows the picture and the display name associated with your profile. You can easily change this by clicking on your name at the top and then editing your profile details,

image

image

The name and picture you set here will be used in the review details section. So you have the control to change this anytime.

Clicking on the ‘Write a review’ button brings up the review submit dialog, (who’s excited about pink buttons! :-)

image

You need to provide a rating, that’s mandatory. The submit button will be disabled until you select a rating. The review comment is optional and you can choose not to enter any text, though I recommend entering the text as it helps the developer get more details out of the review and figure out what you like/dislike about the extension.

After you provide a rating and review comment, click on ‘submit’ and your review will magically appear in the details section!

image

If you see a review that’s offensive or just plain spam, use the flag icon on the review to report it. We have three categories that show up currently,

image

You can select the most relevant option while reporting a review. Our team will run through the reported reviews and take appropriate action based on the content of the review.

That’s it for now, stay tuned for more!

You can read more about this here,

https://blogs.msdn.microsoft.com/bharry/2016/03/23/ratings-and-reviews-in-the-vs-marketplace/

https://blogs.msdn.microsoft.com/visualstudioalm/2016/03/23/now-rate-and-review-extensions-on-marketplace/

We’d love to hear any feedback, feel free to leave a comment or ping me on twitter at @prabhuk

Visual Studio 2015 and .NET 4.6 Released (and more!)

VS2015

Big release day today! Microsoft today announced the release of Visual Studio 2015, Visual Studio 2013 Update 5, TFS 2013 Update 5, .NET 4.6

Check out the blog posts below for more details on what’s new in the release.

The Visual Studio Bloghttp://blogs.msdn.com/b/visualstudio/archive/2015/07/20/visual-studio-2015-and-visual-studio-2013-update-5-released.aspx

Soma’s blog posthttp://blogs.msdn.com/b/somasegar/archive/2015/07/20/visual-studio-2015-and-net-4-6-available-for-download.aspx

ScottGu’s Bloghttp://weblogs.asp.net/scottgu/released-today-visual-studio-2015-asp-net-4-6-asp-net-5-ef-7-previews

.NET Blog – Announcing .NET 4.6http://blogs.msdn.com/b/dotnet/archive/2015/07/20/announcing-net-framework-4-6.aspx

Note that one big guy missing from the list is TFS 2015, it’s still in RC2 and will be RTM’ed very soon.

Take the tools out for a spin and if you have any feedbacks or suggestions send them across using Send-a-Smile, User Voice or the Visual Studio Connect Site