Re: [PATCH v2] dtc: create tool to diff device trees

[David Gibson]

On Thu, Jan 07, 2016 at 11:03:14AM -0800, Frank Rowand wrote:
> From: Frank Rowand <frank.rowand@xxxxxxxxxxxxxx>
> 
> Create script to diff device trees.
> 
> The device tree can be in any of the forms recognized by the dtc compiler:
>   - source
>   - binary blob
>   - file system tree (from /proc/devicetree)
> 
> If the device tree is a source file, then it is pre-processed in the
> same way as it would be when built in the linux kernel source tree
> before diffing.
> 
> Signed-off-by: Frank Rowand <frank.rowand@xxxxxxxxxxxxxx>
> ---
> 
> Tools to develop and debug device tree are somewhat inadequate.  This is a
> small step in improving the situation.
> 
> Rationale for and examples of using the script are provided in slides
> 1 - 78 of the elce 2015 presentation "Solving Device Tree Issues",
> which can be found at:
> 
>    http://elinux.org/images/0/04/Dt_debugging_elce_2015_151006_0421.pdf
> 
> (The script was named dtdiff instead of dtx_diff in the presentation.)
> 
> Changes in v2:
>   - Remove dt-bindings from list of includes in cpp_flags
>   - Remove arch_dtc_flags, which were generated from the arch specific
>     dts makefile
>   - Reformat to 8 character tabs
>   - compile_to_dts(): added back missing return for binary blob
> 
> 
>  scripts/dtc/dtx_diff |  343 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 343 insertions(+)
> 
> Index: b/scripts/dtc/dtx_diff

I think this is probably the wrong directory to put this in.  Because
it preprocesses in the style of the kernel, this script belongs in the
kernel tree, not the upstream dtc tree.  However, basically everything
else in this directory is imported directly from upstream dtc.

Putting this kernel-specific file in here will probably make updates
to newer upstream dtc versions more complicated.

> ===================================================================
> --- /dev/null
> +++ b/scripts/dtc/dtx_diff
> @@ -0,0 +1,343 @@
> +#! /bin/bash
> +
> +# Copyright (C) 2015 Frank Rowand
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; version 2 of the License.
> +
> +
> +usage() {
> +
> +	# use spaces instead of tabs in the usage message
> +	cat >&2 <<eod
> +
> +Usage:
> +
> +   `basename $0` DTx
> +        decompile DTx
> +
> +   `basename $0` DTx_1 DTx_2
> +        diff DTx_1 and DTx_2
> +
> +
> +       -f           print full dts in diff (--unified=99999)
> +       -h           synonym for --help
> +       -help        synonym for --help
> +      --help        print this message and exit
> +       -s SRCTREE   linux kernel source tree is at path SRCTREE
> +                        (default is current directory)
> +       -S           linux kernel source tree is at root of current git repo
> +       -u           unsorted, do not sort DTx
> +
> +
> +Each DTx is processed by the dtc compiler to produce a sorted dts source
> +file.  If DTx is a dts source file then it is pre-processed in the same
> +manner as done for the compile of the dts source file in the Linux kernel
> +build system ('#include' and '/include/' directives are processed).
> +
> +If two DTx are provided, the resulting dts source files are diffed.
> +
> +If DTx is a directory, it is treated as a DT subtree, such as
> +  /proc/device-tree.
> +
> +If DTx contains the binary blob magic value in the first four bytes,
> +  it is treated as a binary blob (aka .dtb or FDT).
> +
> +Otherwise DTx is treated as a dts source file (aka .dts).
> +
> +   If this script is not run from the root of the linux source tree,
> +   and DTx utilizes '#include' or '/include/' then the path of the
> +   linux source tree can be provided by '-s SRCTREE' or '-S' so that
> +   include paths will be set properly.
> +
> +   The shell variable \${ARCH} must provide the architecture containing
> +   the dts source file for include paths to be set properly for '#include'
> +   or '/include/' to be processed.
> +
> +   If DTx_1 and DTx_2 are in different architectures, then this script
> +   may not work since \${ARCH} is part of the include path.  Two possible
> +   workarounds:
> +
> +      `basename $0` \\
> +          <(ARCH=arch_of_dtx_1 `basename $0` DTx_1) \\
> +          <(ARCH=arch_of_dtx_2 `basename $0` DTx_2)
> +
> +      `basename $0` ARCH=arch_of_dtx_1 DTx_1 >tmp_dtx_1.dts
> +      `basename $0` ARCH=arch_of_dtx_2 DTx_2 >tmp_dtx_2.dts
> +      `basename $0` tmp_dtx_1.dts tmp_dtx_2.dts
> +      rm tmp_dtx_1.dts tmp_dtx_2.dts
> +
> +   If DTx_1 and DTx_2 are in different directories, then this script will
> +   add the path of DTx_1 and DTx_2 to the include paths.  If DTx_2 includes
> +   a local file that exists in both the path of DTx_1 and DTx_2 then the
> +   file in the path of DTx_1 will incorrectly be included.  Possible
> +   workaround:
> +
> +      `basename $0` DTx_1 >tmp_dtx_1.dts
> +      `basename $0` DTx_2 >tmp_dtx_2.dts
> +      `basename $0` tmp_dtx_1.dts tmp_dtx_2.dts
> +      rm tmp_dtx_1.dts tmp_dtx_2.dts
> +
> +eod
> +}
> +
> +
> +compile_to_dts() {
> +
> +	dtx="$1"
> +
> +	if [ -d "${dtx}" ] ; then
> +
> +		# -----  input is file tree
> +
> +		if ( ! ${DTC} -I fs ${dtx} ) ; then
> +			exit 3
> +		fi
> +
> +	elif [ -f "${dtx}" ] && [ -r "${dtx}" ] ; then
> +
> +		magic=`hexdump -n 4 -e '/1 "%02x"' ${dtx}`
> +		if [ "${magic}" = "d00dfeed" ] ; then
> +
> +			# -----  input is FDT (binary blob)
> +
> +			if ( ! ${DTC} -I dtb ${dtx} ) ; then
> +				exit 3
> +			fi
> +
> +			return
> +
> +		fi
> +
> +		# -----  input is DTS (source)
> +
> +		if ( cpp ${cpp_flags} -x assembler-with-cpp ${dtx} \
> +			| ${DTC} -I dts ) ; then
> +			return
> +		fi
> +
> +		echo ""                                                      >&2
> +		echo "Possible hints to resolve the above error:"            >&2
> +		echo "  (hints might not fix the problem)"                   >&2
> +
> +		hint_given=0
> +
> +		if [ "${ARCH}" = "" ] ; then
> +			hint_given=1
> +			echo ""                                              >&2
> +			echo "  shell variable \$ARCH not set"               >&2
> +		fi
> +
> +		dtx_arch=`echo "/${dtx}" | sed -e 's|.*/arch/||' -e 's|/.*||'`
> +
> +		if [ "${dtx_arch}" != ""  -a "${dtx_arch}" != "${ARCH}" ] ; then
> +			hint_given=1
> +			echo ""                                              >&2
> +			echo "  architecture ${dtx_arch} is in file path,"   >&2
> +			echo "  but does not match shell variable \$ARCH"    >&2
> +			echo "  (${ARCH}) does not match shell variable"     >&2
> +			echo "  \$ARCH (${ARCH})"                            >&2
> +		fi
> +
> +		if [ ! -d ${srctree}/arch/${ARCH} ] ; then
> +			hint_given=1
> +			echo ""                                              >&2
> +			echo "  ${srctree}/arch/${ARCH}/ does not exist"     >&2
> +			echo "  Is \$ARCH='${ARCH}' correct?"                >&2
> +			echo "  Possible fix: use '-s' option"               >&2
> +
> +			git_root=`git rev-parse --show-toplevel 2>/dev/null`
> +			if [ -d ${git_root}/arch/ ] ; then
> +				echo "  Possible fix: use '-S' option"       >&2
> +			fi
> +		fi
> +
> +		if [ $hint_given = 0 ] ; then
> +			echo ""                                              >&2
> +			echo "  No hints available."                         >&2
> +		fi
> +
> +		echo ""                                                      >&2
> +
> +		exit 3
> +
> +	else
> +		echo ""                                                     >&2
> +		echo "ERROR: ${dtx} does not exist or is not readable"      >&2
> +		echo ""                                                     >&2
> +		exit 2
> +	fi
> +
> +}
> +
> +
> +# -----  start of script
> +
> +cmd_diff=0
> +diff_flags="-u"
> +dtx_file_1=""
> +dtx_file_2=""
> +dtc_sort="-s"
> +help=0
> +srctree=""
> +
> +
> +while [ $# -gt 0 ] ; do
> +
> +	case $1 in
> +
> +	-f )
> +		diff_flags="--unified=999999"
> +		shift
> +		;;
> +
> +	-h | -help | --help )
> +		help=1
> +		shift
> +		;;
> +
> +	-s )
> +		srctree="$2"
> +		shift 2
> +		;;
> +
> +	-S )
> +		git_root=`git rev-parse --show-toplevel 2>/dev/null`
> +		srctree="${git_root}"
> +		shift
> +		;;
> +
> +	-u )
> +		dtc_sort=""
> +		shift
> +		;;
> +
> +	*)
> +		if [ "${dtx_file_1}"  = "" ] ; then
> +			dtx_file_1="$1"
> +		elif [ "${dtx_file_2}" = "" ] ; then
> +			dtx_file_2="$1"
> +		else
> +			echo ""                                             >&2
> +			echo "ERROR: Unexpected parameter: $1"              >&2
> +			echo ""                                             >&2
> +			exit 2
> +		fi
> +		shift
> +		;;
> +
> +	esac
> +
> +done
> +
> +if [ "${srctree}" = "" ] ; then
> +	srctree="."
> +fi
> +
> +if [ "${dtx_file_2}" != "" ]; then
> +	cmd_diff=1
> +fi
> +
> +if (( ${help} )) ; then
> +	usage
> +	exit 1
> +fi
> +
> +# this must follow check for ${help}
> +if [ "${dtx_file_1}" = "" ]; then
> +	echo ""                                                             >&2
> +	echo "ERROR: parameter DTx required"                                >&2
> +	echo ""                                                             >&2
> +	exit 2
> +fi
> +
> +
> +# -----  prefer dtc from linux kernel, allow fallback to dtc in $PATH
> +
> +if [ "${KBUILD_OUTPUT:0:2}" = ".." ] ; then
> +	__KBUILD_OUTPUT="${srctree}/${KBUILD_OUTPUT}"
> +elif [ "${KBUILD_OUTPUT}" = "" ] ; then
> +	__KBUILD_OUTPUT="."
> +else
> +	__KBUILD_OUTPUT="${KBUILD_OUTPUT}"
> +fi
> +
> +DTC="${__KBUILD_OUTPUT}/scripts/dtc/dtc"
> +
> +if [ ! -x ${DTC} ] ; then
> +	__DTC="dtc"
> +	if ( ! which ${__DTC} >/dev/null ) ; then
> +
> +		# use spaces instead of tabs in the error message
> +		cat >&2 <<eod
> +
> +ERROR: unable to find a 'dtc' program
> +
> +   Preferred 'dtc' (built from Linux kernel source tree) was not found or
> +   is not executable.
> +
> +      'dtc' is: ${DTC}
> +
> +      If it does not exist, create it from the root of the Linux source tree:
> +
> +         'make scripts'.
> +
> +      If not at the root of the Linux kernel source tree -s SRCTREE or -S
> +      may need to be specified to find 'dtc'.
> +
> +      If 'O=\${dir}' is specified in your Linux builds, this script requires
> +      'export KBUILD_OUTPUT=\${dir}' or add \${dir}/scripts/dtc to \$PATH
> +      before running.
> +
> +      If \${KBUILD_OUTPUT} is a relative path, then '-s SRCDIR', -S, or run
> +      this script from the root of the Linux kernel source tree is required.
> +
> +   Fallback '${__DTC}' was also not in \${PATH} or is not executable.
> +
> +eod
> +		exit 2
> +	fi
> +	DTC=${__DTC}
> +fi
> +
> +
> +# -----  cpp and dtc flags same as for linux source tree build of .dtb files,
> +#        plus directories of the dtx file(s)
> +
> +dtx_path_1_dtc_include="-i `dirname ${dtx_file_1}`"
> +
> +dtx_path_2_dtc_include=""
> +if (( ${cmd_diff} )) ; then
> +	dtx_path_2_dtc_include="-i `dirname ${dtx_file_2}`"
> +fi
> +
> +cpp_flags="\
> +	-nostdinc                                  \
> +	-I${srctree}/arch/${ARCH}/boot/dts         \
> +	-I${srctree}/arch/${ARCH}/boot/dts/include \
> +	-I${srctree}/drivers/of/testcase-data      \
> +	-undef -D__DTS__"
> +
> +dtc_flags="\
> +	-i ${srctree}/arch/${ARCH}/boot/dts/ \
> +	-i ${srctree}/kernel/dts             \
> +	${dtx_path_1_dtc_include}            \
> +	${dtx_path_2_dtc_include}"
> +
> +DTC="${DTC} ${dtc_flags} -O dts -qq -f ${dtc_sort} -o -"
> +
> +
> +# -----  do the diff or decompile
> +
> +if (( ${cmd_diff} )) ; then
> +
> +	diff ${diff_flags} \
> +		<(compile_to_dts "${dtx_file_1}") \
> +		<(compile_to_dts "${dtx_file_2}")
> +
> +else
> +
> +	compile_to_dts "${dtx_file_1}"
> +
> +fi
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Attachment: signature.asc
Description: PGP signature