Re: [PATCH] perf tools: Fix python extension build

[Josh Boyer]

On Sat, Feb 27, 2016 at 3:21 PM, Jiri Olsa <jolsa@xxxxxxxxxx> wrote:
> On Fri, Feb 26, 2016 at 11:59:03PM +0100, Jiri Olsa wrote:
>> On Fri, Feb 26, 2016 at 05:49:02PM -0500, Josh Boyer wrote:
>>
>> SNIP
>>
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/evsel.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/trace-event.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/xyarray.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/python.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/ctype.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/cgroup.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/rblist.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/string.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/hweight.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/rbtree.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/bitmap.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/find_bit.o
>> > >
>> > > not sure we want to come up with some 'nicer' solution
>> >
>> > I don't think anything under python_ext_build/tmp/ actually matters in
>> > any significant way.  As long as this doesn't negatively impact
>> > something via side-effect, it's probably good enough.
>>
>> right, and it's probably generic sollution too
>>
>> >
>> > Will you write up a full patch?
>>
>> yep, will post it tomorrow
>>
>> jirka
>
> sending full patch
>
> jirka
>
>
> ---
> The util/python-ext-sources file contains sources files
> for python extension relative to $(srctree)/tools/perf,
>
> Such a file path $(FILE).c is handed over to the python
> extension build system, which builts final object in
> $(PYTHON_EXTBUILD)/tmp/$(FILE).o path.
>
> After build is done all files from $(PYTHON_EXTBUILD)lib/
> are carried as the result binaries.
>
> Above system fails when we add source file relative to
> ../lib, which we do for:
>
>   ../lib/bitmap.c
>   ../lib/find_bit.c
>   ../lib/hweight.c
>   ../lib/rbtree.c
>
> All above objects will be built like:
>
>   $(PYTHON_EXTBUILD)/tmp/../lib/bitmap.c
>   $(PYTHON_EXTBUILD)/tmp/../lib/find_bit.c
>   $(PYTHON_EXTBUILD)/tmp/../lib/hweight.c
>   $(PYTHON_EXTBUILD)/tmp/../lib/rbtree.c
>
> which accidentally happens to be final library path:
>   $(PYTHON_EXTBUILD)/lib/
>
> Changing setup.py to pass full paths of source files
> to Extension build class and thus keep all built objects
> under $(PYTHON_EXTBUILD)tmp directory.
>
> Cc: David Ahern <dsahern@xxxxxxxxx>
> Cc: Namhyung Kim <namhyung@xxxxxxxxxx>
> Cc: Peter Zijlstra <a.p.zijlstra@xxxxxxxxx>
> Reported-by: Josh Boyer <jwboyer@xxxxxxxxxxxxxxxxx>
> Link: http://lkml.kernel.org/n/tip-z14ivktbyblnz9ivuo0zau8l@xxxxxxxxxxxxxx
> Signed-off-by: Jiri Olsa <jolsa@xxxxxxxxxx>

The Reported-by: tag should probably be changed to Jeff Bastian
<jbastian@xxxxxxxxxx> if there's still time.  He reported the original
bug.  Maybe also include this in stable for 4.2+?

I'll be putting this into Fedora Rawhide today.  Thanks!

josh

> ---
>  tools/perf/util/setup.py | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/tools/perf/util/setup.py b/tools/perf/util/setup.py
> index 1833103768cb..c8680984d2d6 100644
> --- a/tools/perf/util/setup.py
> +++ b/tools/perf/util/setup.py
> @@ -22,6 +22,7 @@ cflags = getenv('CFLAGS', '').split()
>  # switch off several checks (need to be at the end of cflags list)
>  cflags += ['-fno-strict-aliasing', '-Wno-write-strings', '-Wno-unused-parameter' ]
>
> +src_perf  = getenv('srctree') + '/tools/perf'
>  build_lib = getenv('PYTHON_EXTBUILD_LIB')
>  build_tmp = getenv('PYTHON_EXTBUILD_TMP')
>  libtraceevent = getenv('LIBTRACEEVENT')
> @@ -30,6 +31,9 @@ libapikfs = getenv('LIBAPI')
>  ext_sources = [f.strip() for f in file('util/python-ext-sources')
>                                 if len(f.strip()) > 0 and f[0] != '#']
>
> +# use full paths with source files
> +ext_sources = map(lambda x: '%s/%s' % (src_perf, x) , ext_sources)
> +
>  perf = Extension('perf',
>                   sources = ext_sources,
>                   include_dirs = ['util/include'],
> --
> 2.4.3