Using the kafka dotnet client on Apple Silicon

Kafka clients in different programming languages use a common c++ library, which presumably encapsulates the “real” work. As of writing this, the nuget package hasn’t been updated with the arm64 binaries. This causes issues when trying to target dotnet 6.0, as it no longer uses rosetta to emulate x64. I’ve listed some imperfect options below that you must weigh up for yourself.

Option 1

Build it from source and copy the output to one of the paths that the library looks in. If you already have the tools to build a c++ project then this may be a valid option.

Option 2

Use the workaround described in this github issue: run brew install librdkafka and copy the files to one of the established paths, or the root of the project and set it to copy on build. A downside is that this could potentially conflict with the version that’s expected in the dotnet client provided via the nuget package.

Option 3

Remain on dotnet 5.0 for the time being. This isn’t really a serious option though if you’re working on a team of developers where your setup is the only thing holding back from upgrading.

Option 4

Remove the arm64 version of dotnet and install the x64 version. This may be the preferred way of working until all issues are ironed out with various client libraries. One of the disadvantages is that the tool is installed to a different file path meaning you may want to add an alias to it to use it from the terminal, and may also need to update your IDE. In jetbrains rider the setting is called “.NET CLI executable path”.