imagetest-qemu.bbclass, qemuimage-testlib: add support for more FSTYPES

qemuimage-testlib hardcodes ext3 as fs type. This adds support for more
images types which are supported by runqemu: ext[234]/jffs2/btrfs.

I've skipped (for now) vmdk (which qemu can boot) because:
 - we don't have network on images without connman because of the way
runqemu starts vmdk images (can't pass kernel args for network config)
 - qemuimage-testlib-pythonhelper relies on '192.168' being in the output of
ps to return the pid

(From OE-Core rev: 95b7cafafcaa4dda7328632475003f5778ab95bd)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

runqemu-internal: support for ext2 and ext4 not only ext3

Don't check only for ext3 fstype, we can boot ext2 and ext4 just
as well.

(From OE-Core rev: 8fbf21365fbfab9e3cd36c4eab86fe03efa04e8e)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

scripts/runqemu: add ext4 to the list of extracted extensions

(From OE-Core rev: 702deed71de41ef2e93bc5435e136bf219537d3a)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: cooker: Rename status variable to recipecache

"status" is a relic of very early versions of bitbake, rename it to match what it now
means.

(Bitbake rev: db307fc68773868de2db614a061f51c5c6a90468)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: cooker: Rename confusing buildDepgraph task to handlePrefProviders

(Bitbake rev: a25722175bf61f22961f45dbd356cb6d79864b35)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: cooker: Drop pointless variables

(Bitbake rev: 0d7517db234ef5f6b3eb99822ecb5e86cf48fe2e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: cooker.py: Drop confusing updateCache calls

updateCache() gets called by command.py when needed and needs to be iterated
over. The calls in cooker.py are therefore just plain wrong/confusing now.

(Bitbake rev: 8b7c65bba466bb6773c56849074978ce2c956129)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: cooker: Remove pointless repeated import

(Bitbake rev: e01914a8eaa97a410cb5fe9480f2d96bb6553342)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: cooker: Fix get_bbfiles()

Its clear looking at the code the path argument has never been used and doesn't
work. Tweak the function to be a little more readable.

(Bitbake rev: d7047ceaf29b870ade3257e263e010f65aa8bc6e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: cooker: Separate out collections handling code into its own class

The Cooker class is too large and needs to be split up into different
functional units. Splitting out the collections code into its own class
seems like a good place to start to try and disentangle things.

(Bitbake rev: ca1fcbb6e214c155a05328779d3d326e10c5eac0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: cooker: Drop unused variables from collect_bbfiles()

(Bitbake rev: 67a8e13eed6433bd1e4fab82ece5f6774b42fa00)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: pysh: Say what kind of token isn't implemented

When the shell lexer finds an unrecognized dollar token, the error
message should contain what kind of token it is having problems with.

(Bitbake rev: 1acb9c338018c612db519d2db823c66b567863b9)

Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: hob: assure consistency for image size label

After the image was build, update the image size in the packagelistpage.

[YOCTO #4388]
(Bitbake rev: 4d6d02e61b32d1d710c708774f08cc5e7764415d)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: hob: clicking on "cancel" should reset table sorting to default

Changed the "cancel" callback function for both recipes and packages
list page.

[YOCTO #4333]
(Bitbake rev: 05ebbaa2426716c6254c4a98872020d02d2944bd)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: hob: "Brought in by" sorting should take into account the (+#) information

[YOCTO #4332]
(Bitbake rev: 226a5f2d0947460cb6df9e97f0146b65f96a7f8c)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: hob: sort alphabetically results in groups of recipes/packages

When you search a string, there are 2 groups: recipes/packages starting
with that string, and the rest. These 2 groups are now sorted.

[YOCTO #4111]
(Bitbake rev: 8d2b2a1c97570d3c6b40727f1afa1db8c00882f9)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: Updated information in tooltips

Updated information for "Additional free space" and
"Image basic size" to match the description in the
manual.

[HOB #4354]
[HOB #4355]

(Bitbake rev: d3eac3f104689137c071c91a343ca4558b9d9f17)

Signed-off-by: Andrei Dinu <andrei.adrianx.dinu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

rpm: ensure __mkdir_p matches __mkdir

They differ at times because one is set from the db_cv_path_mkdir and the other is
found during config time for the host (/usr/bin/mkdir), in the macros we should just
use __mkdir for the __mkdir_p variant.

[YOCTO #4452]

(From OE-Core rev: 2ab642056829aef675f75c05b9ac5bbc43c87cd1)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

python: remove the default optimization

We had hard coded python-native and python's default optimization to 1,
which made the "assert" statement didn't work, and removed the "-O/-OO"
(optimization options), the target python had a "-N" option to disable
the default optimization, but the native python didn't.

I think that we can set the environment variable PYTHONOPTIMIZE or use
"python -O" if we need to optimize, but I'm not sure whether we need to
set it by default, it would confuse the user or cause/hide unexpected
problems if the "assert" doesn't work.

[YOCTO #4427]

(From OE-Core rev: 165ed464bbb9bf985dde9d8c15d000809901fff6)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake.conf: update way to set default ROOT_HOME

Use "??=" to set default ROOT_HOME. It can be overwrote by developer
with "?=" in any layer and at same time it also can be overwrote in
local.conf.

(From OE-Core rev: 902d3fa57d4659ee12aac80246dcaca5c45f9d8c)

Signed-off-by: Kang Kai <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

classes/insane: add check for PN in OVERRIDES

If a recipe is named such that its PN value matches something already in
OVERRIDES (such as when PN happens to be the same as MACHINE or DISTRO),
it can have unexpected consequences. Assignments such as
FILES_${PN} = "xyz" effectively turn into FILES = "xyz".

Implements [YOCTO #4288].

(From OE-Core rev: c331f0a5cac765174a1b5de5c12aec7e965d4158)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

classes/buildhistory: track contents of selected files in images

Add a BUILDHISTORY_IMAGE_FILES variable specifying a space-separated
list of files within an image to copy into buildhistory, so that changes
to them can be tracked. Typically this would be used for configuration
files, and by default this includes /etc/passwd and /etc/group, but the
user is free to extend this list by setting the variable in local.conf.

Implements [YOCTO #4154].

(From OE-Core rev: ed6bb6e3db518082d3a9c45d548bc1339be2c5ca)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

buildhistory-diff: improve bad command-line argument handling

* Check for existence of specified buildhistory directory and show a
  proper error message if it doesn't
* Show an error message instead of a traceback with a mangled revision
  if one of the specified git revisions is invalid
* Show usage information if --help is specified
* Write error messages to stderr

Fixes [YOCTO #4313].

(From OE-Core rev: 329edb52e9c23c0956b849a660accf39d44f9d9f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

boot-directdisk/bootimg/image-vmdk: remove more nostamps for image generation

We have removed the nostamps for do_rootfs and do_build, so it seems
reasonable to remove the stamps for do_bootimg, do_bootdirectdisk and
do_vmdkimg.

The current problem is that the do_vmdkimg always re-run, but the
do_rootfs may not, so the code like below in the do_rootfs function
doesn't re-run and cause problems (the symlink exists when the
do_vmdkimg creates the symlink again):

    if d.getVar('IMAGE_LINK_NAME', True):
        cmds += "\trm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.*"

(From OE-Core rev: 45c7dab5d51065c7b24ab5292f8e9d3104eb0626)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

kernel.bbclass: make FILES package-specific

Fix QA warnings seen when using 'traditional' kernel recipes e.g.:

WARNING: QA Issue: .../recipes-kernel/linux/linux_3.0.18.bb: Variable
FILES is set as not being package specific, please fix this.

(From OE-Core rev: 799c16ed317aed7638e264ee2f92e4b722f1b011)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: lib: print is a function in moderm python, covert remaining users

(Bitbake rev: d0b180d868390a464b6799ad90db6bbe0350e158)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: lib: Use modern expcetion syntax

(Bitbake rev: a4a37b6a83faa62f61433122c4583e93e64f7372)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: prserv: Drop StandardError usage

StandardError doesn't exist in python 3, use Exception instead.

(Bitbake rev: 4a40046036493f0cdf0f66487ad5ce083461a5c2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: siggen: Add missing error message parameter

There is clearly a missing parameter from this error message, add it.

(Bitbake rev: d98be5ea69b99fb7934fc3093422f139bc37a1cc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: providers: Remove pointless lambda sort function

This lambda function is equivalent to the default sort used by sorted,
so we can simply remove this. The syntax isn't compatible with python 3.

(Bitbake rev: da8550fc884596222daa3f8794dce1abd01e5612)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: cooker: Remove deprecated operator

The <> operator is deprecated, replace with !=.

(Bitbake rev: 4a43e58dd97ec6ea304eaa727c030973a5bc91b7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: lib: Clean up various file access syntax

Python 3 is stricter about how files are accessed. Specficially:

 * Use open(), not file()
 * Use binary mode for binary files (when checksumming)
 * Use with statements to ensure files get closed
 * Add missing file close statements

(Bitbake rev: 9f08b901375ba640f47596f1bcf43f98a931550f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

eglibc/machine-sdk: Drop SDK_GLIBC_ADDONS

We no longer change GLIBC_ADDONS per target so we no longer need to special
case the SDK settings for it either.

(From OE-Core rev: 3fb0e792600b5adb58bc5fca90d8605c1c76280f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

imagetest-qemu: Fix indentation

(From OE-Core rev: 23ba10c61ff26bffb1a84972eb4e0bfb03561f89)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

class/lib: Fix up various file access methods

There are various bits of cruft that have built up around our file accesses. This patch
cleans some of them up, specifically:

 * Remove pointless "from __builtin__ import file"
 * Use open(), not file()
 * Wrap file usage in a with container to ensure files are closed
 * Add missing .close() calls in some cases

(From OE-Core rev: a43e0a8ecd0441131e929daf998c3cd454d9c8f3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

buildstats: Drop spurious open() call

This file is already opened a few lines previously, so drop the duplicated
call.

(From OE-Core rev: 4808629faa1222b31f92a3e410e06adb8e081293)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

useradd: Use modern join syntax

string.join() is deprecated syntax and doesn't work under python3
so use the modern sytax instead.

(From OE-Core rev: 402d652edf79bcfe1eaf1a3b55ad1177d1e3ee85)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

powertop: inherit gettext

powertop uses gettext during config time and at build time to generate
the messages, so really should inherit gettext

[YOCTO #4470]

(From OE-Core rev: 9eb94ba3d98bf6196eb779590f2fb842ec664177)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

classes/lib: Fix getcmdstatus breakage

I mistakenly thought subprocess had getcmdstatus in python 2. It doesn't so lets
add a wrapper and have this work in both worlds.

(From OE-Core rev: 2253e9f12734c6e6aa489942b5e4628eca1fa29d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: git: remove symling before updating mirror tarball

* with read-only PREMIRROR (e.g. mounted over NFS or CIFS
  and referenced as file:///mnt/premirror) we cannot use
  BB_GENERATE_MIRROR_TARBALLS because all git2_abc.git.tar.gz
  files later became just symlinks to read-only location in PREMIRROR
  (it works fine on first build and for new components, because
  at that time there isn't tarball on PREMIRROR yet).

  ERROR: Fetcher failure: Fetch command failed with exit code 141, output:
  tar (child): /build/downloads/git2_abc.git.tar.gz: Cannot open: Read-only file system
  tar (child): Error is not recoverable: exiting now

(Bitbake rev: 3627b02f77c78beedadadd77c619b9e5edaae076)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: bin/bitbake: add a check for /dev/shm existing

Python multiprocessing needs /dev/shm, and if it doesn't exist (e.g. in
a chroot environment) or is not writable, you'll get
"OSError: [Errno 38] Function not implemented" or
"OSError: [Errno 13] Permission denied" with a traceback, which doesn't
really help to explain what is wrong.

Implements [YOCTO #4407].

(Bitbake rev: e7460d01f281f913a94192e1f4bd20688164bdd4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: utils: Improve better_compile error message

Similarly to the better_exec improvements, improve the compile failure
messages to be more user readable.

(Bitbake rev: 9bc92d0210e13e4cc98727f6c9ec2f47c2221e77)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: utils: Improve better_exec traceback handling

The current bitbake tracebacks are hard to read/confusing and sometimes
incomplete. This patch attempts to do better by:

* Moving the note about the exact exception to the end to make things
  read in sequence
* Merged the initial stack trace to become part of the code dump
* Added handling for "/xxxx" file paths since we can load these files
  and include the data as part of the trace
* Dropped the ERROR: prefix to every line, allowing the error messages to
  be spacially accosicated in the UIs
* Moved the "From:" line to the top of each code block and ensured its present
  consistently

With the complexity now in this funciton, I've added try/except wrapping around
it to ensure we catch exceptions in the exception handler too.

Example before:

"""
ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/eglibc/eglibc-initial_2.17.bb:
TypeError: 'filter' object is not subscriptable

ERROR: The stack trace of python calls that resulted in this exception/failure was:
ERROR:   File "do_populate_lic", line 13, in <module>
ERROR:
ERROR:   File "do_populate_lic", line 6, in do_populate_lic
ERROR:
ERROR:   File "license.bbclass", line 99, in find_license_files
ERROR:
ERROR:   File "/media/build1/poky/meta/lib/oe/license.py", line 38, in visit_string
ERROR:     if pos > 0 and license_pattern.match(elements[pos-1]):
ERROR:
ERROR: The code that was being executed was:
ERROR:      0009:    destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True))
ERROR:      0010:    copy_license_files(lic_files_paths, destdir)
ERROR:      0011:
ERROR:      0012:
ERROR:  *** 0013:do_populate_lic(d)
ERROR:      0014:
ERROR: [From file: 'do_populate_lic', lineno: 13, function: <module>]
ERROR:      0002:def do_populate_lic(d):
ERROR:      0003:    """
ERROR:      0004:    Populate LICENSE_DIRECTORY with licenses.
ERROR:      0005:    """
ERROR:  *** 0006:    lic_files_paths = find_license_files(d)
ERROR:      0007:
ERROR:      0008:    # The base directory we wrangle licenses to
ERROR:      0009:    destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True))
ERROR:      0010:    copy_license_files(lic_files_paths, destdir)
ERROR: [From file: 'do_populate_lic', lineno: 6, function: do_populate_lic]
ERROR:      0095:        lic_files_paths.append((os.path.basename(path), srclicfile))
ERROR:      0096:
ERROR:      0097:    v = FindVisitor()
ERROR:      0098:    try:
ERROR:  *** 0099:        v.visit_string(license_types)
ERROR:      0100:    except oe.license.InvalidLicense as exc:
ERROR:      0101:        bb.fatal('%s: %s' % (d.getVar('PF', True), exc))
ERROR:      0102:    except SyntaxError:
ERROR:      0103:        bb.warn("%s: Failed to parse it's LICENSE field." % (d.getVar('PF', True)))
ERROR: [From file: 'license.bbclass', lineno: 99, function: find_license_files]
ERROR: Function failed: do_populate_lic
ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/i586-poky-linux/eglibc-initial/2.17-r3/temp/log.do_populate_lic.17442
"""

Example after:

"""
ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/eglibc/eglibc-initial_2.17.bb:

The stack trace of python calls that resulted in this exception/failure was:
File: 'do_populate_lic', lineno: 13, function: <module>
     0009:    destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True))
     0010:    copy_license_files(lic_files_paths, destdir)
     0011:
     0012:
 *** 0013:do_populate_lic(d)
     0014:
File: 'do_populate_lic', lineno: 6, function: do_populate_lic
     0002:def do_populate_lic(d):
     0003:    """
     0004:    Populate LICENSE_DIRECTORY with licenses.
     0005:    """
 *** 0006:    lic_files_paths = find_license_files(d)
     0007:
     0008:    # The base directory we wrangle licenses to
     0009:    destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True))
     0010:    copy_license_files(lic_files_paths, destdir)
File: 'license.bbclass', lineno: 99, function: find_license_files
     0095:        lic_files_paths.append((os.path.basename(path), srclicfile))
     0096:
     0097:    v = FindVisitor()
     0098:    try:
 *** 0099:        v.visit_string(license_types)
     0100:    except oe.license.InvalidLicense as exc:
     0101:        bb.fatal('%s: %s' % (d.getVar('PF', True), exc))
     0102:    except SyntaxError:
     0103:        bb.warn("%s: Failed to parse it's LICENSE field." % (d.getVar('PF', True)))
File: '/media/build1/poky/meta/lib/oe/license.py', lineno: 38, function: visit_string
     0034:        new_elements = []
     0035:        elements = filter(lambda x: x.strip(), license_operator.split(licensestr))
     0036:        for pos, element in enumerate(elements):
     0037:            if license_pattern.match(element):
 *** 0038:                if pos > 0 and license_pattern.match(elements[pos-1]):
     0039:                    new_elements.append('&')
     0040:                element = '"' + element + '"'
     0041:            elif not license_operator.match(element):
     0042:                raise InvalidLicense(element)
Exception: TypeError: 'filter' object is not subscriptable

ERROR: Function failed: do_populate_lic
ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/i586-poky-linux/eglibc-initial/2.17-r3/temp/log.do_populate_lic.3275
ERROR: Task 9 (/media/build1/poky/meta/recipes-core/eglibc/eglibc-initial_2.17.bb, do_populate_lic) failed with exit code '1
"""

(Bitbake rev: c5de66b870406d9bd1161a9b7e2b04fe6eb065fe)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: bin/bitbake: Ensure early errors are shown to the user

Currently if errors occur early in the init process, the errors may
not be shown to the user. This change ensures that if a failure does
occur, the messages are flushed from the queue and shown to the user.

(Bitbake rev: fda84c4285e0bc31c249b6dd5464aeb6ad622a9a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake.conf: Use casting to ensure valid comparision

python3 is stricter about type comparisions so add an explicit cast to int()
to ensure this code is portable.

(From OE-Core rev: b9a1b9ad55c0f9fec082ffa37e576d8fd664becd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

base.bbclass: Drop unused/obsolete import

(From OE-Core rev: e501a98e87b8854040ac4081326871d7d6397917)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

image/kernel-module-split/eglibc-ld.inc: Remove has_key() usage

The has_key() attribute has been removed in python 3 since there is better
syntax available. Use the improved syntax.

(From OE-Core rev: 3dff13793e875ff58cc38c4a960caca9b6969843)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

sanity/patch.py: Remove commands module usage

The commands module is removed in python3. Use the subprocess module instead
and the pipes module to replace the mkargs usage.

(From OE-Core rev: e2e1dcd74bc45381baccf507c0309dd792229afe)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

insane: Open file in binary mode

We open the file we're writing to in binary mode so open the input stream with the
same mode so things match. This avoids errors with python3.

(From OE-Core rev: 4f47b3a4726dd47e8a6db228fcaf25d1890e3e52)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

meta/lib/oe: Replace StandardError with Exception

StandardError is removed in python3, replace with Exception class instead.

(From OE-Core rev: a37ae30b9766df346ca57755530024a0b7d5f86b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

classes/lib: Update to use print as a function call

In python 3 print is a function call. In some cases bb.note is a more
appropriate call to make.

(From OE-Core rev: 754874e68d1037c76fc4cc7d35d3fa12946d00f8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

metadata_scm: Replace deprecated operator

The <> operator is deprecated, use != instead which is equivalent.

(From OE-Core rev: dde7af5f6c769eae721b1ee18462b841c8ea3277)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

sanity.bbclass: Use open(), not file()

file() syntax is removed in python 3, use open() instead.

(From OE-Core rev: b2d10f15db23246e3957b69d77433f87674928bb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

classes/recipes/lib: Fix various python whitespace issues

There are some left over tab characters in the python functions. This
removes them and resolves python 3 errors.

(From OE-Core rev: fafeb381c48291fa65c634c01c244843c8d7fad3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

classes: Use modern exception raising syntax

Modern expection rasing syntax is function call format, convert to this
to keep python 3 happy and model correct coding style in the core.

(From OE-Core rev: f4b382754603d3f1caa13824bcc8d06b568bbc59)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

buildstats: Update to ensure files are closed

python3 doesn't like files being left open. This updates the code style
to ensure file are closed.

(From OE-Core rev: ec74285bd2108f12f33fc8ac0dc1d124ab48be21)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

clases/lib: Use modern exception syntax

Update older code to use modern exception handling syntax which
is the form accepted by python 3.

(From OE-Core rev: b010501cd089e649a68f683be0cf4d0aac90fbe3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

core-image-minimal-initramfs: Remove odd quote characters

The quote characters present trigger python 3 characters type warnings,
we don't need them so replace them with normal quote characters.

(From OE-Core rev: 3c104443506cb89d72944e46096a94a80838a707)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

xserver-xorg: Add xkbcomp to RRECOMMENDS

OE-Core commit bdcc5e8f1286d288baf410458efc39a59b68d751 removed xkbcomp
from RDEPENDS for xkeyboard-config but X server still needs it otherwise
it fails to start.

(From OE-Core rev: f2330ebc3071d780cbc6d1ddab5c54bfadf8fffc)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

gcc-configure-cross.inc: Remove info generation

Some distros have newer makeinfo which does not go well with
anything below gcc 4.8, this fix avoids regenerating info files

(From OE-Core rev: 50ac2ed5299e2b47b2f3fd9c9cde9d733d2f8d9b)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bootimg.bbclass: add comment for NOHDD

Add comment for NOHDD which is used for skipping building the HDDIMG if
set to 1

(From OE-Core rev: 9843ad9d783f68b97fedfe5b435528538bb26c1c)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

gnutls: upgrade to 2.12.23

Importantly, this fixes CVE-2013-1619.

Upstream doesn't use GNU as a host, so update the SRC_URI.

remove-gets.patch isn't required anymore, obsolete_automake_macros.patch is
merged upstream, and gnutls_fix_for_automake_1.12.1.patch doesn't seem to be
needed.  It was merged and reverted upstream, and my build without it succeeded.

(From OE-Core rev: 9a6395076984350b1dd7005453f97233bbb43132)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

oprofileui(-server): upgraded to GIT revision f168b8bf

'obsolete_automake_macros.patch' is not needed anymore, because it
was integrated in the specified commit.

(From OE-Core rev: 4c7120a24cef2ff6b9df821c69153363c938ac77)

Signed-off-by: Bogdan Marinescu <bogdan.a.marinescu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

qt4: remove trailing slash from prefix

The trailing slashs on "Prefix=${prefix}/" and "-prefix ${prefix}/" are passed
through to the generated pkgconfig files and may be joined to paths like
"/include" yielding a final path with a double forward-slash (eg.
"/usr//include"). This may end up in the debugging symbols in other applications
or libraries which depend on qt4 which in turn causes the debugedit program to
fail with the message "canonicalization unexpectedly shrank by one character"
when it tries to replace the double forward-slash with a single forward-slash.
Thus the function split_and_strip_files fails and ultimately do_package fails.

As this slash is removed from the prefix it is added into the regular expression
used to fix up pkgconfig files later in the recipe.

This error was seen in vlc in meta-openembedded and should be solved by this
change in openembedded-core.

(From OE-Core rev: 61a7329a5c1f0b84f447256134f5a77917f07f38)

Signed-off-by: Paul Barker <paul@paulbarker.me.uk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

gdb: tabs -> spaces

(From OE-Core rev: 8ec07ae2ebfd97aa1413a3c3be0dc612eee7e929)

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

gdb: AArch64 lacks lttng support

(From OE-Core rev: 2288c2a9abe828ffb0bbd527105fb633ac818354)

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

craklib:fix testnum and teststr failed

Error log:
...
$ ./testnum
(null).pwd.gz: No such file or directory
PWOpen: No such file or directory

$ ./util/teststr
(null).pwd.gz: No such file or directory
PWOpen: No such file or directory
...

Set DEFAULT_CRACKLIB_DICT as the path of  PWOpen

(From OE-Core rev: 9c78d2ef5291b29fa313f9d4bf6a81c395207d69)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

cracklib: fix default dictionary should be generated for target endianness

The previous dict files are NOT byte-order independent, in fact they are
probably ARCHITECTURE SPECIFIC.
Create the dict files in big endian, and convert to host endian while
load them. This could fix the endian issue on multiple platform.

[Bug #4419]

(From OE-Core rev: 075d5a19bb56b0bf492e7dc7f453c2e23139104b)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

linux-yocto/3.8: udpate to v3.8.11

Update the linux-yocto-3.8 SRCREVs to the korg 3.8.11 -stable release.

(From OE-Core rev: b60d04331f84e06fae6c074acab045c3e003929b)

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

linux-yocto/3.4: update to v3.4.43, ltsi-65311d8 and v3.4.42-rt56

Updating the 3.4 kernel trees to the latest korg, LTSI and -rt stable
releases.

(From OE-Core rev: 98446039e210936a32e70559a379131e8e0d832f)

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

kern-tools: anchor and delimit regexs

Updating the kern-tools SRCREV with the following fix:

    Updateme is responsible for updating an existing meta-series with new patches,
    configs and tree manipulations. To do this, it first checks for an existing
    board description and generates one if required. It then searches for features
    and fragments to be applied for the tree.

    There were two problems:

     - A top level board description is detected via the presence of "define"
       directives that indicate the board name, the arch and kernel type. The
       test for define would match on patches or fragments with 'define' in their
       name, and would incorrectly use that file as the top level board description.

       This is fixed by ensuring that only defines at the start of a line, or preceded
       by whitepace match.

     - When searching for features that were indicated as 'addon' or 'optional', the
       search would find, and apply, any feature with the passed name as substring
       versus an exact match.

       This is fixed by ensuring that the matched feature name is /<feature name>
       versus <feature name>

(From OE-Core rev: 57ae1e412a35d827f84bf9b1f48747bf703f84b7)

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

linux-yocto/3.4: iwlwifi, ext4, brtfs warning and -rt fixes

Updating the yocto 3.4 SRCREVs for the following fixes:

206d4bb powerpc/perf: run irq-work under softirq context in rt
8a969f9 arm/perf: run irq-work under softirq context in rt
79ba946 iwlwifi: fix unused variable warning
59d93fa ext4: remove unused variable in ext4_update_super()
2385eee Btrfs: fix compile warnings in extent_io.c

(From OE-Core rev: 96113b28031583af1dd42fe7660a9d5ced36d1b6)

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

dmidecode: fix fetch path to use BP

This is so the multilib prefix is not used

(From OE-Core rev: 50e26e553152485461d096463db0fb4f4fa826d6)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

gcc-configure-target: Add --enable-dependency-tracking to EXTRA_OECONF

Fixes errors seen while compiling gcc 4.8 for target, errors like

| /home/khem/work/angstrom/build/tmp-angstrom_next-eglibc/work-shared/gcc-4.8.0-r0/gcc-4.8.0/libatomic/cas_n.c:125:1: fatal error: opening dependency file .deps/cas_1_.lo.Ppo: No such file or directory

which are due to disabled dependency tracking but dependency files being still generated
while compiling libatomics.

(From OE-Core rev: c83cbc80af078d7be5deec44bba6e0ca29a5ee1a)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

tcmode-default.inc: Pin binutils to 2.23.2

(From OE-Core rev: 27759800b020ad52c2e91b8eea3acf9216860a0a)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

binutils: backport whitespace fix

This is a backport of a whitespace skipping issue: PR14987 and PR14887

(From OE-Core rev: f4312eaeb6eb2b7196be2b968e2e468e705fb2cf)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

binutils: Upgrade 2.23.1 -> 2.23.2

(From OE-Core rev: 202f46b139ab20079e97cdeb147d59e23bcdfe01)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

guile: Update to 2.0.9

fix_parallel_build_issue.patch is now part of upstream.

(From OE-Core rev: 87cb30d988843ab1159d23789cd334aaf2a77e34)

Signed-off-by: Jonathan Liu <net147@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

qemu: Ensure kernel nfsd module is enabled

This will ensure that qemu images that include the nfs-server package have the kernel
feature correctly enabled

(From OE-Core rev: 57c718c6288f2a2538173cdd3d401d70f939a40a)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

gdb-cross: gdb-cross requires readline-native

gdb depends on readline so gdb-cross needs to depend on readline-native to
build successfully.

(From OE-Core rev: ee132d68220d7f515760d47db57e00d1d8263a1a)

Signed-off-by: Mike Crowe <mac@mcrowe.com>
Acked-by: Phil Blundell <philb@gnu.org>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

libgnome-keyring: remove, nothing in oe-core uses this anymore

(From OE-Core rev: d5f985f51b250b88dd910ca3a1fc8f2992271e60)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

gnome-keyring: remove, nothing uses it in oe-core

(From OE-Core rev: fbefbf60fd18dae135b96d0bfd2db832f568fea9)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

libsoup: upgrade to 2.42.1

Upgrade to the latest stable release, and fix the dependencies.

gnome-keyring use has been removed from libsoup, and sqlite3 is now a hard
dependency.  The optional libsoup-gnome library is now a small stub, so enable
it by default but allow it to be disabled via PACKAGECONFIG.

(From OE-Core rev: 3f093e24e78cc8a553bebc03ec9f3a32700981e6)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

packagegroup-core-x11-sato: Add udev-extraconf to maintain compatibility

Since we have removed udev-extraconf as a RRECOMMENDS in udev
add it back here to maintain compatibility

(From OE-Core rev: 8e2df16e9ab7433a868da2a3db4a25441b92d0ed)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

packagegroup-core-lsb: Add udev-extraconf for LSB compatibility

(From OE-Core rev: ce6927b0d662c8929f514c92bb0e96c54bf47446)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

udev: do not recommend udev-extraconf

same as already sent by Koen Kooi [1]

[1] http://patches.openembedded.org/patch/48183/

(From OE-Core rev: b190d9d692bde12cce1062bd0cba65a5b7fc8188)

Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

systemd: do not recommend udev-extraconf for udev

* adding this extra functionality is an image- or distro-decision
* at least automount-rule/script breaks other automount solutions causing
  misleading kernel messages as 'VFS: could not find a valid V7 on sda'

same as already sent by Koen Kooi [1]

[1] http://patches.openembedded.org/patch/48139/

(From OE-Core rev: 860cd2f9ba5803824ab59bf9c0e84b83f7907dc1)

Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

qemux86: disable paravirt guest in linux 3.4, causes test failures

The shutdown sanity test that we previously saw on qemux86-64 in 2.6.37 have
started to re-appear reproducably on qemux86 in 3.4 in the Danny series.  Until
this can be root-caused and fixed in the kernel, disable paravirtualisation
across all kernels and all qemu machines.

[ YOCTO #4196 ]

(From OE-Core rev: e20c3bf6ce0d8ab5653739dd02555186b78b8912)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

archive-*-source.bbclass: handle dependency in anonymous function

Using "before do_rootfs" would have this task added to do_rootfs
dependencies regardless of the filtering options. Instead, add this
dependency in the anonymous python function.

(From OE-Core rev: 2400a74923e51e17ecfa94c2d63908b0b2aac76a)

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

cronie: Convert crontab to a system user

crontab is meant to be a system user, not a normal user of the system
so lets pass the correct parameters to the system. If we don't do this
it interferes with the setup of normal users, particularly if some
preset list of users is desired.

(From OE-Core rev: ea6b28cad959d9ebbaf0ecbd695d1d72c0c01151)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

grub-efi-native: Cleanup whitespace

A few of the recent commits have used inconsistent whitespace with
respect to tabs and spaces for indentation and alignment. Correct it.

(From OE-Core rev: 3472c1f7ab409cd91c1d4782d9e00880b84e3ae8)

Signed-off-by: Darren Hart <dvhart@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

neard: Rename the recipe as we are already in 0.10

(From OE-Core rev: 0f26f430d2bcca37eea63740dae51ac1e49bc550)

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

buildhistory: change default BUILDHISTORY_DIR to TOPDIR

* like PERSISTENT_DIR I find BUILDHISTORY_DIR more persistent
  then tmp-eglibc directory
* WARN: people with existing checkouts in TMPDIR/buildhistory
  need to move it to new location first (in order to save history)

(From OE-Core rev: d3ad9a4c573fec08b5606a4c3e86f93cb95e8718)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: fetch2/svn: Do not save authentication

(Bitbake rev: a9b1e7c2ad13ff47acb629bae082d4f96e174f99)

Signed-off-by: Simon Vanveerdeghem <simon.vanveerdeghem@barco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: bitbake: remove stack trace depth limitation

In case of an error in bitbake, the stack trace is limited to 5
items. This is an endless source of confusion and it makes bugs
reports impractical, since a full stack trace can't be included
in the bug report. This patch simply removes the depth limit.

(Bitbake rev: 02629c42fb09413d9da16cfe43e03338ce7db3ff)

Signed-off-by: Bogdan Marinescu <bogdan.a.marinescu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: data_smart.py: add some internal bitbake variables to configuration hash

Take __BBTASKS, __BBHANDLERS and __BBANONFUNCS into account when
computing the configuration hash.

[YOCTO #4447]

(Bitbake rev: 260ced7452405fc43ce3d9dd6798236aa07cc716)

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: prserv/cooker: Handle PRService errors cleanly

Current if the PR Service fails to start, bitbake carries on regardless or
hangs with no error message. This adds an exception and then handles it correctly
so the UIs correctly handle the error and exit cleanly.

[YOCTO #4010]

(Bitbake rev: 949c01228a977c3b92bfc0802f6c71b40d8e05b3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: goggle: fix invalid error check

runCommand was returning 'error' but checking for 'err'
resulting in an exception.

(Bitbake rev: 263aa44ed47e0cb03e1a97e7ed4a50bd9f828c78)

Signed-off-by: Martin Donnelly <martin.donnelly@ge.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

bitbake: Update to version 1.19.0 for master

(Bitbake rev: c47088a86fd8ad06c1810d04d9537c4cd01e8bef)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

ia32-base.inc: remove inapropriate grub dependency

There is no good reason for ia32 machines to have hard dependency on grub,
as there are other bootloaders available for ia32 platforms.

(From OE-Core rev: d03c0c24704c6ab6d2cfcf9bf705f6ace2a247cc)

Signed-off-by: Tomas Frydrych <tomas@sleepfive.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>