Nuitka takes extremely long time to compile

See original GitHub issue

Problem: I tried to compile a file, Nuitka spent way more time than I expected, and process bar stuck at 301/301, while CPU utilization is 0%. After about half hour, I’ve to use Ctrl-C to manually end the process. To my surprise, after I enter Ctrl-C it magically continue and finish its compiling process.

SOFTWARE ENVIRONMENT Windows10 21H2 Nuitka 0.6.19.4 Python 3.9.10

HARDWARE ENVIRONMENT Ryzen7 1700 32GB DDR4-2666 SN750 500G

COMMAND

nuitka --mingw64 --standalone --plugin-enable=multiprocessing --plugin-enable=pyqt5 --plugin-enable=numpy --plugin-enable=tk-inter --include-module=matplotlib.pyplot --output-dir=C:\Users\home\Desktop\Python\1 C:\Users\Home\PycharmProjects\gambling_simulator\main.py

OUT-PUT

Nuitka:INFO: Completed Python level compilation and optimization.
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: gcc (gcc).
Nuitka-Scons:INFO: Slow C compilation detected, used 60s so far, this might indicate scalability problems.
Nuitka-Scons:INFO: Running 'C:\\Users\\Home\\AppData\\Local\\Nuitka\\Nuitka\\gcc\\x86_64\\11.2.0-12.0.1-9.0.0-r1\\mingw64\\bin\\gcc.exe -o "C:\\Users\\home\\Desktop\\Python\\1\\MAIN~1.DIS\\main.exe" -fuse-linker-plugin -flto=16 -fpartial-inlining -freorder-functions -Wl,--exclude-all-symbols -Wl,--out-implib,.\\import.lib -municode -O3 -Wl,--stack,9863168 -Wl,--enable-auto-import -s -static-libgcc @".\\@link_input.txt" -LC:\\Users\\home\\AppData\\Local\\Programs\\Python\\Python39\\libs -lm -lpython39' took 1725.83 seconds
Nuitka-Scons:INFO: Compiled 301 C files using ccache.
Nuitka-Scons:INFO: Cached C files (using ccache) with result 'cache miss': 294
Nuitka-Scons:INFO: Cached C files (using ccache) with result 'cache hit': 7
Nuitka-Plugins:INFO: pyqt5: Including Qt plugins 'iconengines,imageformats,mediaservice,platforms,platformthemes,printsupport,styles' below 'PyQt5\qt-plugins'.
Nuitka-Plugins:INFO: pyqt5: Found 2 files DLLs from 'OpenSSL' PyQt5installation.
Nuitka-Plugins:INFO: numpy: Found 1 file DLLs from '' numpyinstallation.
Nuitka-Plugins:INFO: data-files: Included data file 'lib2to3\Grammar3.9.10.final.0.pickle' due to package data for 'lib2to3.pgen2'.
Nuitka-Plugins:INFO: data-files: Included data file 'lib2to3\PatternGrammar3.9.10.final.0.pickle' due to package data for 'lib2to3.pgen2'.
Nuitka-Plugins:INFO: matplotlib: Included data dir 'matplotlib\\mpl-data' with 183 files due to: package data for 'matplotlib.
Nuitka-Plugins:INFO: matplotlib: Included data file 'matplotlib\mpl-data\matplotlibrc' due to Updated matplotlib config file with backend to use..
Nuitka:INFO: Keeping build directory 'C:\\Users\\home\\Desktop\\Python\\1\\main.build'.
Nuitka:INFO: Successfully created 'C:\\Users\\home\\Desktop\\Python\\1\\main.dist\\main.exe'.

Code

from matplotlib import pyplot
import random
import time
import multiprocessing
import os


class coin():
    def __init__(self,balance,bet_rate,repeat):
        self.balance = balance
        self.bet_rate = bet_rate
        self.repeat = repeat

    def get_probability(self):
        s = time.perf_counter()
        x_axis, y_axis, tasks = [], [], []
        distribution = self.get_tasks_balanced_complexity(1,self.repeat+1,os.cpu_count()-1)
        pool = multiprocessing.Pool(processes=len(distribution))
        for x in distribution:
            tasks.append(pool.apply_async(self.worker,args=(x[0],x[1])))
        pool.close()
        pool.join()
        for x in tasks:
            x_a, y_a = x.get()
            x_axis += x_a
            y_axis += y_a
        e = time.perf_counter()
        print(round(e - s, 2), 's')
        pyplot.plot(x_axis, y_axis, color='#000000')
        pyplot.show()

    def worker(self,start,end):
        x_axis, y_axis = [], []
        for x in range(start,end):
            #print('Worker-'+str(os.getpid())+':',str(round((x-start)/(end-start),2)*100)+'%')
            win = 0
            for y in range(10000):
                balance = self.run_simulation(self.balance, x)
                if balance > self.balance:
                    win += 1
            y_axis.append(win/10000)
            x_axis.append(x)
        return x_axis, y_axis

    def run_simulation(self,balance,repeat):
        for x in range(repeat):
            bet = self.bet_rate * balance
            balance -= bet
            while True:
                R = random.random()
                if R > 0.5:
                    balance += bet * 2
                    break
                elif R < 0.5:
                    break
        return balance

    @staticmethod
    def get_tasks_balanced_complexity(start,end,number):
        task = []
        sum = (start + end - 1) * (end - start) / 2
        complexity = sum / number
        while start < end:
            for x in range(start, end):
                if ((start + x) * (x - start + 1) / 2) / complexity >= 1.0:
                    task.append((start, x))
                    break
                elif ((start + end - 1) * (end - start) / 2) <= complexity:
                    task.append((start, end))
                    break
            if task[-1][0] != task[-1][1]:
                start = task[-1][1]
            else:
                start = task[-1][1] + 1
        print(task)
        return task


if __name__ == '__main__':
    coin(1000,0.05,int(input())).get_probability()

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:13 (6 by maintainers)

github_iconTop GitHub Comments

2reactions
kayhayencommented, Jan 28, 2022

I would try anti-bloat first, then maybe --lto=no, and definitely using MSVC as well, as it’s often more scalable.

0reactions
kayhayencommented, Aug 8, 2022

@huangxin168 posting below closed issues is a surefire way to get ignored.

Read more comments on GitHub >

github_iconTop Results From Across the Web

python 3.x - How make nuitka compile faster? - Stack Overflow
You know, compiling stuff can take time. There are projects for which the whole compilation can take hours. It's, a priori, no clear...
Read more >
Nuitka-chat/community - Gitter
submodules, takes all night ́(about ́12 houts) in my setup. I also use Nuitka to compile some fat frameworks, like Django, bundled with...
Read more >
I would imagine Nuitka is slower, because it cannot apply ...
Yes. A jit compiler can compile based on both code and data, and can compile all the dependencies. In contrast an aot compiler...
Read more >
Buiding Nuitka: A Compiler Written in Python - Morioh
In this tutorial, we'll share Nuitka is the Python compiler. ... with the same input files, will take a long time and much...
Read more >
Python compiler nuitka - Reddit
It takes some elbow grease to get your build environment and commands just right, but the package control line help menu is super...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found