.NET6 isolated function on Windows fails

See original GitHub issue

I have a pair of functions in a .net 6.0 v4 project. When running locally they both work as expected - one is a queue trigger and the other is a timer trigger. But when I zip deploy to a function app in Azure, ~only the queue triggered function works. The timer triggered function fails immediately with the following error:~

EDIT: on further investigation, it doesn’t appear that the queue function was working either. I’m starting to think that it’s something about the configuration of the app itself.

Type : System.InvalidOperationException
Total Count : 9
Message : Did not find any initialized language workers

The full callstack is here:

Full Exception :
 Exception while executing function: Functions.PcfReader ---> System.InvalidOperationException : Did not find any initialized language workers
   at Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcherLoadBalancer.GetLanguageWorkerChannel(IEnumerable`1 rpcWorkerChannels) at /_/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcherLoadBalancer.cs : 27
   at async Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.InvokeAsync(ScriptInvocationContext invocationContext) at /_/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcher.cs : 389
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker.InvokeCore(Object[] parameters,FunctionInvocationContext context) at /_/src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs : 95
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) at /_/src/WebJobs.Script/Description/FunctionInvokerBase.cs : 82
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Host.Executors.VoidTaskMethodInvoker`2.InvokeAsync[TReflected,TReturnType](TReflected instance,Object[] arguments) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\VoidTaskMethodInvoker.cs : 20
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2.InvokeAsync[TReflected,TReturnValue](Object instance,Object[] arguments) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.cs : 52
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeWithTimeoutAsync(IFunctionInvoker invoker,ParameterHelper parameterHelper,CancellationTokenSource timeoutTokenSource,CancellationTokenSource functionCancellationTokenSource,Boolean throwOnTimeout,TimeSpan timerInterval,IFunctionInstance instance) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 596
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstanceEx instance,ParameterHelper parameterHelper,ILogger logger,CancellationTokenSource functionCancellationTokenSource) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 542
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance,FunctionStartedMessage message,FunctionInstanceLogEntry instanceLogEntry,ParameterHelper parameterHelper,ILogger logger,CancellationToken cancellationToken) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 320
   End of inner exception
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance,FunctionStartedMessage message,FunctionInstanceLogEntry instanceLogEntry,ParameterHelper parameterHelper,ILogger logger,CancellationToken cancellationToken) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 367
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.TryExecuteAsync(IFunctionInstance functionInstance,CancellationToken cancellationToken) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 108

I can provide the function name and subscription details privately if necessary, thank you.

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:10 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
msft-mwcommented, Apr 5, 2022

This turned out to be an issues with indirectly referenced old packages, configuration, and the publishing profile, all of which were only discoverable in the Kudu logs. The issue can be considered resolved, but some information from the team about if or how the issue could’ve been seen in the portal would be very helpful for future readers.

Issue 1: It was not possible to find any compatible framework version This was caused by Visual Studio’s default publishing profile having a Deployment mode setting of Framework-dependent.

Resolution Changing the deployment mode in the publishing profile to Self-contained

Issue 2: Did not find any initialized language workers This ended up being a combination of two things:

  • a configuration setting was missing that instructed the AzureServiceTokenProvider to use the user-assigned identity
  • the Azure Key Vault configuration extension package Microsoft.Extensions.Configuration.AzureKeyVault pulled in an older version of Microsoft.Azure.Services.AppAuthentication (1.0.3) which did not support the documented connection string.

Resolution Adding the configuration string and then explicitly (rather than indirectly) referencing the most recent version of Microsoft.Azure.Services.AppAuthentication

There is a newer library for Key Vault integration but we haven’t had time to investigate usage at this time, or compatibility with functions.

1reaction
msft-mwcommented, Apr 13, 2022

repro.zip

Repro instructions

  1. Download and unzip the attached respro code.

  2. Create a function app with the following settings:

    • net6.0
    • v4
    • dotnet-isolated
    • windows
  3. Create a user managed identity and assign it to the function. The function should not have a system assigned identity enabled, only the user managed identity.

  4. Create a new key vault instance with RBAC-based access control.

  5. Assign the user identity from step 3 using the Key Vault Reader role.

  6. In the Program.cs, replace the AppId in the string on L28 with the Client Id from the user managed identity in step 3.

  7. On line 32, replace the first parameter with the URL of the key vault created in step 5.

  8. Publish the function to the function app using the following settings (I did this from Visual Studio 2022):

    • Configuration: Release|AnyCPU
    • Target framework: net6.0
    • Deployment mode: Framework-dependent
    • Target runtime: win-x64
  9. Once the function starts up, it should start failing immediately. In Application Insights, System.InvalidOperation exceptions should start showing up with the message Exception while executing function: Functions.Repro Did not find any initialized language workers

  10. Going to the Kudu debug console and navigating to the logs under C:\home\LogFiles\Application\Functions\Host will show the error:

    2022-04-13T19:54:21.318 [Information] The framework 'Microsoft.NETCore.App', version '6.0.4' (x64) was not found.
    2022-04-13T19:54:21.318 [Information] - The following frameworks were found:
    2022-04-13T19:54:21.318 [Information] 2.2.14 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
    2022-04-13T19:54:21.318 [Information] 3.0.3 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
    2022-04-13T19:54:21.318 [Information] 3.1.18 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
    2022-04-13T19:54:21.319 [Information] 3.1.23 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
    2022-04-13T19:54:21.319 [Information] 5.0.14 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
    2022-04-13T19:54:21.319 [Information] 5.0.15 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
    2022-04-13T19:54:21.319 [Information] 6.0.2 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
    2022-04-13T19:54:21.319 [Information] 6.0.3 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
    

    Observation: The framework error only started appearing once I added the key vault extensions nuget package. Prior to that, the function actually ran as expected.

  11. From the publish profile, change the Deployment mode setting to Self-contained and republish the function.

  12. Identical errors will appear in Application Insights as in step 10.

  13. Again going to the logs from the Kudu debug console, the error changes to:

    2022-04-13T20:04:25.536 [Information] Unhandled exception. System.ArgumentException: Connection string RunAs=App;AppId=REDACTED is not valid. Must contain 'TenantId' attribute and it must not be empty.
    2022-04-13T20:04:25.537 [Information] at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProviderFactory.ValidateAttribute(Dictionary`2 connectionSettings, String attribute, String connectionString)
    2022-04-13T20:04:25.538 [Information] at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProviderFactory.Create(String connectionString, String azureAdInstance)
    2022-04-13T20:04:25.538 [Information] at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider..ctor(String connectionString, String azureAdInstance)
    2022-04-13T20:04:25.538 [Information] at NoLanguageRepro.Program.<>c.<Main>b__0_0(HostBuilderContext _, IConfigurationBuilder c) in C:\Users\REDACTEDMYUSERNAME\Documents\Visual Studio 2022\Projects\NoLanguageRepro\NoLanguageRepro\Program.cs:line 28
    2022-04-13T20:04:25.538 [Information] at Microsoft.Extensions.Hosting.HostBuilder.BuildAppConfiguration()
    2022-04-13T20:04:25.538 [Information] at Microsoft.Extensions.Hosting.HostBuilder.Build()
    2022-04-13T20:04:25.539 [Information] at NoLanguageRepro.Program.Main() in C:\Users\mwan\Documents\Visual Studio 2022\Projects\NoLanguageRepro\NoLanguageRepro\Program.cs:line 23
    
Read more comments on GitHub >

github_iconTop Results From Across the Web

Did not find functions with language [dotnet-isolated].
However, in our production subscription, the functions 'intermittently' report the error "Did not find functions with language [dotnet-isolated] ...
Read more >
Guide for running C# Azure Functions in an isolated worker ...
Learn how to use a .NET isolated worker process to run your C# functions in Azure, which supports non-LTS versions of .NET and...
Read more >
Running dotnet-isolated AzureFunction does not work
One of the workaround to resolve this issue: Created Azure Functions (Stack: .Net 6 Isolated) in Visual Studio and run locally:.
Read more >
Can't run/debug Azure Function isolated (V4) .NET in Rider ...
Can't run/debug Azure Function isolated (V4) .NET in Rider when using Launch Settings profile. Fails to set the host Follow.
Read more >
How to Change an Azure Function Written in .NET from In ...
Login to the Azure Portal and change the FUNCTIONS_WORKER_RUNTIME Configuration value from dotnet to dotnet-isolated. After clicking OK, don't ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found