This is an issue with microsoft's default debian docker image. There is something with the TLS handshake that doesn't pass with some versions of SQL Server. The solution is to change to an Ubuntu image (debian/alpine both have this issue):

...
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-bionic AS base
...
FROM mcr.microsoft.com/dotnet/core/sdk:3.1-bionic AS build
...