139 replies to this topic

[neo]

Done: https://github.com/O...e9d543588faf121

Edited by neo, 2 May 2023 - 14:01.

[frankviana]

Thx.

[frankviana]

Do you know why I see only until 52 threads?

 

I understand now...

 

Show 52, but running all 72 threads (I think it's running normally, but it just doesn't show the other 20 threads)

 

 

 CPU.png   620.3KB   0 downloads

Edited by frankviana, 2 May 2023 - 14:47.

[neo]

There is a limit to how many tasks you can run in parallel, eventually you either run out of tasks all together, or you run out of non-dependent tasks (things that have to wait until others are finished).

 

Besides that, you have to be carefull with defining max BB threads and max parallel make, with a bit of bad luck bitbake starts 20 threads each with a compile with 20 threads, and your build crashes (maxing out the CPU, overheating it, running out of memory).

 

On massive machines, it it recommended that BB_NUMBER_THREADS is not set higher than 32, and PARALLEL_MAKE not higher than 20.

 

And personally I think that is a bit much, I've noticed that the performance and duration of the build flattens out if you set them too high.

[frankviana]

Done: https://github.com/O...e9d543588faf121

Initialising tasks: 100% |#########################################################| Time: 0:00:07
Sstate summary: Wanted 1674 Local 0 Network 0 Missed 1674 Current 550 (0% match, 24% complete)
NOTE: Executing Tasks
ERROR: openresolv-3.5.2-r0 do_fetch: Fetcher failure for URL: 'git://github.com/NetworkConfiguration/openresolv.git;protocol=https'. Please set a valid SRCREV for url ['SRCREV_default_pn-openresolv', 'SRCREV_default', 'SRCREV_pn-openresolv', 'SRCREV'] (possible key names are git://github.com/NetworkConfiguration/openresolv.git;protocol=https, or use a ;rev=X URL parameter)
ERROR: Logfile of failure stored in: /home/frank/OpenPLi/openpli-oe-core/build/tmp/work/all-oe-linux/openresolv/3.5.2-r0/temp/log.do_fetch.9473
ERROR: Task (/home/frank/OpenPLi/openpli-oe-core/meta-openpli/recipes-connectivity/openresolv/openresolv_3.5.2.bb:do_fetch) failed with exit code '1'
NOTE: Tasks Summary: Attempted 2631 tasks of which 2588 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /home/frank/OpenPLi/openpli-oe-core/meta-openpli/recipes-connectivity/openresolv/openresolv_3.5.2.bb:do_fetch
Summary: There was 1 WARNING message shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
make: *** [Makefile:131: image] Erro 1
frank@frank-X99:~/OpenPLi/openpli-oe-core$

 

[neo]

Chips, typo, sorry. https://github.com/O...bc63709682ad2f9

[mrdude]

 

Done: https://github.com/O...e9d543588faf121

Initialising tasks: 100% |#########################################################| Time: 0:00:07
Sstate summary: Wanted 1674 Local 0 Network 0 Missed 1674 Current 550 (0% match, 24% complete)
NOTE: Executing Tasks
ERROR: openresolv-3.5.2-r0 do_fetch: Fetcher failure for URL: 'git://github.com/NetworkConfiguration/openresolv.git;protocol=https'. Please set a valid SRCREV for url ['SRCREV_default_pn-openresolv', 'SRCREV_default', 'SRCREV_pn-openresolv', 'SRCREV'] (possible key names are git://github.com/NetworkConfiguration/openresolv.git;protocol=https, or use a ;rev=X URL parameter)
ERROR: Logfile of failure stored in: /home/frank/OpenPLi/openpli-oe-core/build/tmp/work/all-oe-linux/openresolv/3.5.2-r0/temp/log.do_fetch.9473
ERROR: Task (/home/frank/OpenPLi/openpli-oe-core/meta-openpli/recipes-connectivity/openresolv/openresolv_3.5.2.bb:do_fetch) failed with exit code '1'
NOTE: Tasks Summary: Attempted 2631 tasks of which 2588 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /home/frank/OpenPLi/openpli-oe-core/meta-openpli/recipes-connectivity/openresolv/openresolv_3.5.2.bb:do_fetch
Summary: There was 1 WARNING message shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
make: *** [Makefile:131: image] Erro 1
frank@frank-X99:~/OpenPLi/openpli-oe-core$

 

I get the same error, it's down to this file:

openpli-oe-core/meta-openpli/recipes-connectivity/openresolv/openresolv_3.5.2.bb

 

It's trying to download from a url that no longer exists - from http://roy.marples.name/downloads/....

 

The bb file needs to be updated to point to a url where you can get openresolve-3.5.2-r0.tar.bz and the md5 sum and sha will probably also need udated if getting from elsewhere.

[neo]

Is in the pipeline. This change I could do remotely without having to test it first.

[mrdude]

Is in the pipeline. This change I could do remotely without having to test it first.

Do you mean pointing to the url where openresolv exists?

[neo]

No, that has already been done, I thought the question was related to upgrading the version.

[mrdude]

No, that has already been done, I thought the question was related to upgrading the version.

I tried it about 20 minutes ago and it still failed with openresolv error, I ended up changing the SRC line in the bb file to a different url and it then worked.

Edited by mrdude, 2 May 2023 - 18:16.

[neo]

I'll be home in 30 minutes or so, then I can test.

[mrdude]

I'll be home in 30 minutes or so, then I can test.

No worries, I just changed the SRC line to this:

 

SRC_URI = "https://dev.alpinelinux.org/archive/${BPN}/${BP}.tar.bz2 \

 

Then the image built fine.

[neo]

Fixed.

 

Switching to some other tarball is not done when the source repo is available.

[mrdude]

Fixed.

 

Switching to some other tarball is not done when the source repo is available.

Thanks for fixing.

[frankviana]

All cores  

 

 Captura de tela de 2023-05-02 23-22-24.png   614.67KB   0 downloads

 

 

Edited by frankviana, 3 May 2023 - 03:23.

[frankviana]

octagonemmc

 

 Captura de tela de 2023-05-03 00-15-26.png   69.03KB   0 downloads

Initialising tasks: 100% |################################################################| Time: 0:00:06
Sstate summary: Wanted 340 Local 325 Network 0 Missed 15 Current 1884 (95% match, 99% complete)
Removing 8 stale sstate objects for arch sf8008: 100% |###################################| Time: 0:00:00
Removing 2 stale sstate objects for arch cortexa15hf-neon-vfpv4: 100% |###################| Time: 0:00:00
NOTE: Executing Tasks
ERROR: openpli-enigma2-image-1.0-r0 do_image_octagonemmc: ExecutionError('/home/frank/OpenPLi/openpli-oe-core/build/tmp/work/sf8008-oe-linux-gnueabi/openpli-enigma2-image/1.0-r0/temp/run.do_image_octagonemmc.2510008', 127, None, None)
ERROR: Logfile of failure stored in: /home/frank/OpenPLi/openpli-oe-core/build/tmp/work/sf8008-oe-linux-gnueabi/openpli-enigma2-image/1.0-r0/temp/log.do_image_octagonemmc.2510008
Log data follows:
| DEBUG: Executing python function set_image_size
| DEBUG: 370479.200000 = 284984 * 1.300000
| DEBUG: 524288.000000 = max(370479.200000, 524288)[524288.000000] + 0
| DEBUG: 524288.000000 = int(524288.000000)
| DEBUG: 524288 = aligned(524288)
| DEBUG: returning 524288
| DEBUG: Python function set_image_size finished
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['virtual:native:/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-core/update-rc.d/update-rc.d_0.8.bb:do_populate_sysroot', 'virtual:native:/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-devtools/opkg/opkg_0.4.4.bb:do_populate_sysroot', '/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-core/glibc/ldconfig-native_2.12.1.bb:do_populate_sysroot', '/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-devtools/qemu/qemuwrapper-cross_1.0.bb:do_populate_sysroot', 'virtual:native:/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-devtools/opkg-utils/opkg-utils_0.4.3.bb:do_populate_sysroot', '/home/frank/OpenPLi/openpli-oe-core/meta-openpli/recipes-openpli/enigma2/enigma2.bb:do_populate_sysroot', 'virtual:native:/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.46.1.bb:do_populate_sysroot', 'virtual:native:/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-extended/parted/parted_3.4.bb:do_populate_sysroot', '/home/frank/OpenPLi/openpli-oe-core/meta-octagon/recipes-bsp/octagon-buildimage-native_1.0.bb:do_populate_sysroot', 'virtual:native:/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-devtools/dosfstools/dosfstools_4.2.bb:do_populate_sysroot', 'virtual:native:/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-devtools/mtools/mtools_4.0.26.bb:do_populate_sysroot', 'virtual:native:/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-extended/pigz/pigz_2.6.bb:do_populate_sysroot', '/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-kernel/kmod/depmodwrapper-cross_1.0.bb:do_populate_sysroot', 'virtual:native:/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-extended/zip/zip_3.0.bb:do_populate_sysroot', '/home/frank/OpenPLi/openpli-oe-core/meta-octagon/recipes-linux/linux-octagon_4.4.35.bb:do_populate_sysroot', 'virtual:native:/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-devtools/makedevs/makedevs_1.0.1.bb:do_populate_sysroot', 'virtual:native:/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot', '/home/frank/OpenPLi/openpli-oe-core/openembedded-core/meta/recipes-core/glibc/cross-localedef-native_2.33.bb:do_populate_sysroot']
| NOTE: Installed into sysroot: []
| NOTE: Skipping as already exists in sysroot: ['update-rc.d-native', 'opkg-native', 'ldconfig-native', 'qemuwrapper-cross', 'opkg-utils-native', 'enigma2', 'e2fsprogs-native', 'parted-native', 'octagon-buildimage-native', 'dosfstools-native', 'mtools-native', 'pigz-native', 'depmodwrapper-cross', 'zip-native', 'linux-octagon', 'makedevs-native', 'pseudo-native', 'cross-localedef-native', 'xz-native', 'glib-2.0-native', 'shadow-native', 'quilt-native', 'qemu-native', 'gdk-pixbuf-native', 'debianutils-native', 'openssl-native', 'libtool-native', 'automake-native', 'libarchive-native', 'libsolv-native', 'pkgconfig-native', 'autoconf-native', 'kmod-native', 'patch-native', 'perl-native', 'fribidi', 'glibc', 'libjpeg-turbo', 'libudfread', 'libxml2', 'gcc-runtime', 'python3-pillow', 'freetype', 'libmad', 'giflib', 'python3-twisted', 'openssl', 'libsigc++-3', 'python3', 'libdvbsi++', 'python3-wifi', 'tuxtxt-enigma2', 'libpng', 'libdreamdvd', 'avahi', 'shared-mime-info-native', 'attr-native', 'gettext-minimal-native', 'texinfo-dummy-native', 'util-linux-native', 'ncurses-native', 'readline-native', 'unzip-native', 'zlib-native', 'ninja-native', 'libpcre-native', 'gettext-native', 'libffi-native', 'meson-native', 'gtk-doc-native', 'python3-native', 'libjpeg-turbo-native', 'libpng-native', 'gobject-introspection-native', 'lzo-native', 'bzip2-native', 'cmake-native', 'expat-native', 'gnu-config-native', 'm4-native', 'gdbm-native', 'db-native', 'linux-libc-headers', 'zlib', 'libgcc', 'tiff', 'openjpeg', 'lcms', 'libid3tag', 'opkg-utils', 'mm-common', 'libnsl2', 'libxcrypt', 'gdbm', 'readline', 'xz', 'util-linux', 'bzip2', 'libtirpc', 'libffi', 'sqlite3', 'libtuxtxt', 'libdvdnav', 'libdaemon', 'base-files', 'gobject-introspection', 'dbus', 'libcap', 'shadow-sysroot', 'shadow', 'expat', 'update-rc.d', 'base-passwd', 'initscripts', 'glib-2.0', 'itstool-native', 'xmlto-native', 'libxml2-native', 'util-linux-libuuid-native', 'libcap-ng-native', 'libpcre2-native', 're2c-native', 'python3-setuptools-native', 'libnsl2-native', 'sqlite3-native', 'libtirpc-native', 'flex-native', 'curl-native', 'ncurses', 'libcap-ng', 'bash-completion', 'libpam', 'util-linux-libuuid', 'libdvdread', 'autoconf-archive', 'libpcre', 'libxslt-native', 'docbook-xml-dtd4-native', 'docbook-xsl-stylesheets-native', 'cracklib', 'flex']
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_image_octagonemmc
| 60+0 records in
| 60+0 records out
| 61440 bytes (61 kB, 60 KiB) copied, 0.000371241 s, 165 MB/s
| /home/frank/OpenPLi/openpli-oe-core/build/tmp/work/sf8008-oe-linux-gnueabi/openpli-enigma2-image/1.0-r0/temp/run.do_image_octagonemmc.2510008: 151: /usr/bin/fakeroot: not found
| WARNING: exit code 127 from a shell command.
| ERROR: ExecutionError('/home/frank/OpenPLi/openpli-oe-core/build/tmp/work/sf8008-oe-linux-gnueabi/openpli-enigma2-image/1.0-r0/temp/run.do_image_octagonemmc.2510008', 127, None, None)
ERROR: Task (/home/frank/OpenPLi/openpli-oe-core/meta-openpli/recipes-openpli/images/openpli-enigma2-image.bb:do_image_octagonemmc) failed with exit code '1'
NOTE: Tasks Summary: Attempted 6846 tasks of which 6804 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /home/frank/OpenPLi/openpli-oe-core/meta-openpli/recipes-openpli/images/openpli-enigma2-image.bb:do_image_octagonemmc
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
make: *** [Makefile:131: image] Erro 1
frank@frank-X99:~/OpenPLi/openpli-oe-core$

Edited by frankviana, 3 May 2023 - 04:16.

[neo]

You don't have the fakeroot package installed in your distro.

[frankviana]

You don't have the fakeroot package installed in your distro.

 

Do you know other package missing?

 

I found the post with only this packages:

 

sudo apt install -y autoconf automake bison bzip2 cvs default-jre diffstat flex g++ gawk gcc gettext git git-lfs gzip help2man make ncurses-bin lib32ncurses-dev libc6-dev libtool lz4 texinfo patch perl pkg-config subversion tar texi2html zlib1g-dev chrpath libxml2-utils xsltproc libglib2.0-dev python3-setuptools libc6-i386 genromfs guile-2.2-libs quilt zstd

 

 

Thanks.

[neo]

https://docs.yoctopr...quirements.html doesn't mention it as well, but our buildservers have it installed:

$ apt list fakeroot
Listing... Done
fakeroot/jammy,now 1.28-1ubuntu1 amd64 [installed]

and my own server has it installed too

$ dnf list installed fakeroot
Installed Packages
fakeroot.x86_64                          1.30.1-1.el8                              @epel

Edited by neo, 3 May 2023 - 19:45.