1 Protocol Buffers/gRPC Integration Into .NET Build
2 =================================================
4 With Grpc.Tools package version 1.17 we made it easier to compile .proto files
5 in your project using the `dotnet build` command, Visual Studio, or command-line
6 MSBuild. You need to configure the .csproj project according to the way you want
7 to integrate Protocol Buffer files into your build. If you are upgrading an
8 existing project, read through this list of common scenarios and decide if any
9 one of them matches your approach. The protoc command line migration is
10 explained near the end of this document; this migration may be the quickest but
11 not the long-term solution.
13 There is also a Reference section at the end of the file.
18 First thing first, if you found a bug in this new build system, or have a
19 scenario that is not easily covered, please open an [issue in the gRPC
20 repository](https://github.com/grpc/grpc/issues), and **tag the user @kkm000**
21 somewhere in the text (for example, include `/cc @kkm000` at end of the issue
22 text) to seize his immediate attention.
27 ### I just want to compile .proto files into my library
29 This is the approach taken by the examples in the `csharp/examples` directory.
30 Protoc output files (for example, `Helloworld.cs` and `HelloworldGrpc.cs`
31 compiled from `helloworld.proto`) are placed among *object* and other temporary
32 files of your project, and automatically provided as inputs to the C# compiler.
33 As with other automatically generated .cs files, they are included in the source
34 and symbols NuGet package, if you build one.
36 Simply reference your .proto files in a `<Protobuf>` item group. The following
37 example will add all .proto files in a project and all its subdirectories
38 (excluding special directories such as `bin` and `obj`):
42 <Protobuf Include="**/*.proto" />
46 You must add a reference to the NuGet packages Grpc.Tools and Grpc (the latter
47 is a meta-package, in turn referencing Grpc.Core and Google.Protobuf packages).
48 It is **very important** to mark Grpc.Tools as a development-only dependency, so
49 that the *users* of your library do not fetch the tools package:
51 * "Classic" .csproj with `packages.config` (Visual Studio, Mono): This is
52 handled automatically by NuGet. See the attribute added by Visual Studio to the
53 [packages.config](../../examples/csharp/HelloworldLegacyCsproj/Greeter/packages.config#L6)
54 file in the HelloworldLegacyCsproj/Greeter example.
56 * "SDK" .csproj (Visual Studio, `dotnet new`): Add an attribute
57 `PrivateAssets="All"` to the Grpc.Tools package reference. See an example in the
58 [Greeter.csproj](../../examples/csharp/Helloworld/Greeter/Greeter.csproj#L10)
59 example project in this repository. If adding a package reference in Visual
60 Studio, edit the project file and add this attribute. [This is a bug in NuGet
61 client](https://github.com/NuGet/Home/issues/4125).
63 If building a NuGet package from your library with the nuget command line tool
64 from a .nuspec file, then the spec file may (and probably should) reference the
65 Grpc metapackage, but **do not add a reference to Grpc.Tools** to it. .NET "SDK"
66 projects handle this automatically when called from `dotnet pack` by excluding
67 any packages with private assets, such as thus marked Grpc.Tools.
69 #### Per-file options that can be set in Visual Studio
71 For a "classic" project, you can only add .proto files with all options set to
72 default (if you find it necessary to modify these options, then hand-edit the
73 .csproj file). Click on the "show all files" button, add files to project, then
74 change file type of the .proto files to "Protobuf" in the Properties window
75 drop-down. This menu item will appear after you import the Grpc.Tools package:
77 ![Properties in a classic project](doc/integration.md-fig.1-classic.png)
79 For an "SDK" project, you have more control of some frequently used options.
80 **You may need to open and close Visual Studio** for this form to appear in the
81 properties window after adding a reference to Grpc.Tools package (we do not know
82 whether this is a bug or by design, but it looks like a bug):
84 ![Properties in an SDK project](doc/integration.md-fig.2-sdk.png)
86 You can also change options of multiple files at once by selecting them in the
87 Project Explorer together.
89 See the Reference section at end of this file for options that can be set
90 per-file by modifying the source .csproj directly.
92 #### My .proto files are in a directory outside the project
94 Refer to the example files
95 [RouteGuide.csproj](../../examples/csharp/RouteGuide/RouteGuide/RouteGuide.csproj#L58-L60)
96 and [Greeter.csproj](../../examples/csharp/Helloworld/Greeter/Greeter.csproj#L11)
97 in this repository. For the files to show up in Visual Studio properly, add a
98 `Link` attribute with just a filename to the `<Protobuf>` item. This will be the
99 display name of the file. In the `Include` attribute, specify the complete path
100 to file. A relative path is based off the project directory.
102 Or, if using Visual Studio, add files _as links_ from outside directory. In the
103 Add Files dialog, there is a little [down arrow near the Open
104 button](https://stackoverflow.com/a/9770061). Click on it, and choose "Add as
105 link". If you do not select this option, Visual Studio will copy files to the
106 project directory instead.
108 ### I just want to generate proto and gRPC C# sources from my .proto files (no C# compile)
110 Suppose you want to place generated files right beside each respective source
111 .proto file. Create a .csproj library file in the common root of your .proto
112 tree, and add a reference to Grpc.Tools package (this works in Windows too, `$`
113 below stands for a command prompt in either platform):
116 /myproject/myprotofiles$ dotnet new classlib
118 Restoring packages for /myproject/myprotofiles/myprotofiles.csproj...
120 /myproject/myprotofiles$ rm *.cs <-- remove all *.cs files from template;
121 C:\myproject\myprotofiles> del *.cs /y <-- on Windows, use the del command instead.
122 /myproject/myprotofiles$ dotnet add package Grpc.Tools
125 (the latter command also accepts an optional `--version X.Y` switch for a
126 specific version of package, should you need one). Next open the generated
127 .csproj file in a text editor.
129 Since you are not building a package, you may not worry about adding
130 `PrivateAssets="All"` attribute, but it will not hurt, in case you are
131 repurposing the project at some time later. The important part is (1) tell the
132 gRPC tools to select the whole directory of files; (2) order placement of each
133 output besides its source, and (3) not compile the generated .cs files. Add the
134 following stanza under the `<Project>` xml node:
138 <Protobuf Include="**/*.proto" OutputDir="%(RelativeDir)" CompileOutputs="false" />
142 The `Include` tells the build system to recursively examine project directory
143 and its subdirectories (`**`) include all files matching the wildcard `*.proto`.
144 You can instead selectively include your files or selectively exclude files from
145 the glob pattern; [MSBuild documentation explains
146 that](https://docs.microsoft.com/visualstudio/msbuild/msbuild-items). The
147 `OutputDir="%(RelativeDir)"` orders the output directory for each .cs file be
148 same as the corresponding .proto directory. Finally, `CompileOutputs="false"`
149 prevents compiling the generated files into an assembly.
151 Note that an empty assembly is still generated, but you should ignore it. As
152 with any build system, it is used to detect out-of-date dependencies and
155 #### I am getting a warning about a missing expected file!
157 When we are preparing compile, there is no way to know whether a given proto
158 file will produce a *Grpc.cs output or not. If the proto file has a `service`
159 clause, it will; otherwise, it won't, but the build script cannot know that in
160 advance. When we are treating generated .cs files as temporary, this is ok, but
161 when generating them for you, creating empty files is probably not. You need to
162 tell the compiler which files should be compiled with gRPC services, and which
163 only contain protobuffer message definitions.
165 One option is just ignore the warning. Another is quench it by setting the
166 property `Protobuf_NoWarnMissingExpected` to `true`:
170 <Protobuf_NoWarnMissingExpected>true</Protobuf_NoWarnMissingExpected>
174 For a small to medium projects this is sufficient. But because of a missing
175 output dependency, the corresponding .proto file will be recompiled on every
176 build. If your project is large, or if other large builds depend on generated
177 files, and are also needlessly recompiled, you'll want to prevent these rebuilds
178 when files have not in fact changed, as follows:
180 ##### Explicitly tell protoc for which files it should use the gRPC plugin
182 You need to set the `Protobuf` item property `GrpcServices` to `None` for those
183 .proto inputs which do not have a `service` declared (or, optionally, those
184 which do but you do not want a service/client stub for). The default value for
185 the `GrpcServices` is `Both` (both client and server stub are generated). This
186 is easy enough to do with glob patterns if your files are laid out in
187 directories according to their service use, for example:
191 <Protobuf Include="**/*.proto" OutputDir="%(RelativeDir)"
192 CompileOutputs="false" GrpcServices="None" />
193 <Protobuf Update="**/hello/*.proto;**/bye/*.proto" GrpcServices="Both" />
197 In this sample, all .proto files are compiled with `GrpcServices="None"`, except
198 for .proto files in subdirectories on any tree level named `hello/` and `bye`,
199 which will take `GrpcServices="Both"` Note the use of the `Update` attribute
200 instead of `Include`. If you write `Include` by mistake, the files will be added
201 to compile *twice*, once with, and once without GrpcServices. Pay attention not
204 Another example would be the use of globbing if your service .proto files are
205 named according to a pattern, for example `*_services.proto`. In this case, The
206 `Update` attribute can be written as `Update="**/*_service.proto"`, to set the
207 attribute `GrpcServices="Both"` only on these files.
209 But what if no patterns work, and you cannot sort a large set of .proto file
210 into those containing a service and those not? As a last resort,
212 ##### Force creating empty .cs files for missing outputs.
214 Naturally, this results in a dirtier compiler output tree, but you may clean it
215 using other ways (for example, by not copying zero-length .cs files to their
216 final destination). Remember, though, that the files are still important to keep
217 in their output locations to prevent needless recompilation. You may force
218 generating empty files by setting the property `Protobuf_TouchMissingExpected`
223 <Protobuf_TouchMissingExpected>true</Protobuf_TouchMissingExpected>
227 #### But I do not use gRPC at all, I need only protobuffer messages compiled
229 Set `GrpcServices="None"` on all proto files:
233 <Protobuf Include="**/*.proto" OutputDir="%(RelativeDir)"
234 CompileOutputs="false" GrpcServices="None" />
238 #### That's good so far, but I do not want the `bin` and `obj` directories in my tree
240 You may create the project in a subdirectory of the root of your files, such as,
241 for example, `.build`. In this case, you want to refer to the proto files
242 relative to that `.build/` directory as
246 <Protobuf Include="../**/*.proto" ProtoRoot=".."
247 OutputDir="%(RelativeDir)" CompileOutputs="false" />
251 Pay attention to the `ProtoRoot` property. It needs to be set to the directory
252 where `import` declarations in the .proto files are looking for files, since the
253 project root is no longer the same as the proto root.
255 Alternatively, you may place the project in a directory *above* your proto root,
256 and refer to the files with a subdirectory name:
260 <Protobuf Include="proto_root/**/*.proto" ProtoRoot="proto_root"
261 OutputDir="%(RelativeDir)" CompileOutputs="false" />
265 ### Alas, this all is nice, but my scenario is more complex, -OR-
266 ### I'll investigate that when I have time. I just want to run protoc as I did before.
268 One option is examine our [.targets and .props files](Grpc.Tools/build/) and see
269 if you can create your own build sequence from the provided targets so that it
270 fits your needs. Also please open an issue (and tag @kkm000 in it!) with your
271 scenario. We'll try to support it if it appears general enough.
273 But if you just want to run `protoc` using MsBuild `<Exec>` task, as you
274 probably did before the version 1.17 of Grpc.Tools, we have a few build
275 variables that point to resolved names of tools and common protoc imports.
276 You'll have to roll your own dependency checking (or go with a full
277 recompilation each time, if that works for you), but at the very least each
278 version of the Tools package will point to the correct location of the files,
279 and resolve the compiler and plugin executables appropriate for the host system.
280 These property variables are:
282 * `Protobuf_ProtocFullPath` points to the full path and filename of protoc executable, e. g.,
283 "C:\Users\kkm\.nuget\packages\grpc.tools\1.17.0\build\native\bin\windows\protoc.exe".
285 * `gRPC_PluginFullPath` points to the full path and filename of gRPC plugin, such as
286 "C:\Users\kkm\.nuget\packages\grpc.tools\1.17.0\build\native\bin\windows\grpc_csharp_plugin.exe"
288 * `Protobuf_StandardImportsPath` points to the standard proto import directory, for example,
289 "C:\Users\kkm\.nuget\packages\grpc.tools\1.17.0\build\native\include". This is
290 the directory where a declaration such as `import "google/protobuf/wrappers.proto";`
291 in a proto file would find its target.
293 Use MSBuild property expansion syntax `$(VariableName)` in your protoc command
294 line to substitute these variables, for instance,
297 <Target Name="MyProtoCompile">
299 <ProtoCCommand>$(Protobuf_ProtocFullPath) --plugin=protoc-gen-grpc=$(gRPC_PluginFullPath) -I $(Protobuf_StandardImportsPath) ....rest of your command.... </ProtoCCommand>
301 <Message Importance="high" Text="$(ProtoCCommand)" />
302 <Exec Command="$(ProtoCCommand)" />
306 Also make sure *not* to include any file names to the `Protobuf` item
307 collection, otherwise they will be compiled by default. If, by any chance, you
308 used that name for your build scripting, you must rename it.
310 ### What about C++ projects?
312 This is in the works. Currently, the same variables as above are set to point to
313 the protoc binary, C++ gRPC plugin and the standard imports, but nothing else.
314 Do not use the `Protobuf` item collection name so that your project remains
315 future-proof. We'll use it for C++ projects too.
320 ### Protobuf item metadata reference
322 The following metadata are recognized on the `<Protobuf>` items.
324 | Name | Default | Value | Synopsis |
325 |----------------|-----------|----------------------|----------------------------------|
326 | Access | `public` | `public`, `internal` | Generated class access |
327 | ProtoCompile | `true` | `true`, `false` | Pass files to protoc? |
328 | ProtoRoot | See notes | A directory | Common root for set of files |
329 | CompileOutputs | `true` | `true`, `false` | C#-compile generated files? |
330 | OutputDir | See notes | A directory | Directory for generated C# files |
331 | GrpcOutputDir | See notes | A directory | Directory for generated stubs |
332 | GrpcServices | `both` | `none`, `client`, | Generated gRPC stubs |
333 | | | `server`, `both` | |
338 For files _inside_ the project cone, `ProtoRoot` is set by default to the
339 project directory. For every file _outside_ of the project directory, the value
340 is set to this file's containing directory name, individually per file. If you
341 include a subtree of proto files that lies outside of the project directory, you
342 need to set this metadatum. There is an example in this file above. The path in
343 this variable is relative to the project directory.
346 The default value for this metadatum is the value of the property
347 `Protobuf_OutputPath`. This property, in turn, unless you set it in your
348 project, will be set to the value of the standard MSBuild property
349 `IntermediateOutputPath`, which points to the location of compilation object
350 outputs, such as "obj/Release/netstandard1.5/". The path in this property is
351 considered relative to the project directory.
354 Unless explicitly set, will follow `OutputDir` for any given file.
357 Sets generated class access on _both_ generated message and gRPC stub classes.
359 `grpc_csharp_plugin` command line options
362 Under the hood, the `Grpc.Tools` build integration invokes the `protoc` and `grpc_csharp_plugin` binaries
363 to perform code generation. Here is an overview of the available `grpc_csharp_plugin` options:
365 | Name | Default | Synopsis |
366 |---------------- |-----------|----------------------------------------------------------|
367 | no_client | off | Don't generate the client stub |
368 | no_server | off | Don't generate the server-side stub |
369 | internal_access | off | Generate classes with "internal" visibility |
370 | lite_client | off | Generate client stubs that inherit from "LiteClientBase" |
372 Note that the protocol buffer compiler has a special commandline syntax for plugin options.
375 protoc --plugin=protoc-gen-grpc=grpc_csharp_plugin --csharp_out=OUT_DIR \
376 --grpc_out=OUT_DIR --grpc_opt=lite_client,no_server \
377 -I INCLUDE_DIR foo.proto