Jani Nikula [Mon, 1 Aug 2016 09:37:05 +0000 (12:37 +0300)]
Documentation/sphinx: do not warn about missing tools in 'make help'
Simply move the dochelp rule outside of the HAVE_SPHINX check,
overriding the .DEFAULT rule for HAVE_SPHINX=0.
Cc: Jonathan Corbet <corbet@lwn.net> Cc: Christian Kujau <lists@nerdbynature.de> Signed-off-by: Jani Nikula <jani.nikula@intel.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
doc-rst: kernel-doc: fix handling of address_space tags
The RST cpp:function handler is very pedantic: it doesn't allow any
macros like __user on it:
Documentation/media/kapi/dtv-core.rst:28: WARNING: Error when parsing function declaration.
If the function has no return type:
Error in declarator or parameters and qualifiers
Invalid definition: Expecting "(" in parameters_and_qualifiers. [error at 8]
ssize_t dvb_ringbuffer_pkt_read_user (struct dvb_ringbuffer * rbuf, size_t idx, int offset, u8 __user * buf, size_t len)
--------^
If the function has a return type:
Error in declarator or parameters and qualifiers
If pointer to member declarator:
Invalid definition: Expected '::' in pointer to member (function). [error at 37]
ssize_t dvb_ringbuffer_pkt_read_user (struct dvb_ringbuffer * rbuf, size_t idx, int offset, u8 __user * buf, size_t len)
-------------------------------------^
If declarator-id:
Invalid definition: Expecting "," or ")" in parameters_and_qualifiers, got "*". [error at 102]
ssize_t dvb_ringbuffer_pkt_read_user (struct dvb_ringbuffer * rbuf, size_t idx, int offset, u8 __user * buf, size_t len)
------------------------------------------------------------------------------------------------------^
So, we have to remove it from the function prototype.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Commit f42ddca7bebc (doc-rst: kernel-doc directive, fix state machine
reporter) from Marcus Heiser provides a better fix, so this configuration
change is no longer needed.
Jonathan Corbet [Wed, 20 Jul 2016 22:43:41 +0000 (16:43 -0600)]
docs: deprecate kernel-doc-nano-HOWTO.txt
Now that the new Sphinx world order is taking over, the information in
kernel-doc-nano-HOWTO.txt is outmoded. I hate to remove it altogether,
since it's one of those files that people expect to find. But we can add a
warning and fix all the other pointers to it.
Reminded-by: Daniel Vetter <daniel.vetter@ffwll.ch> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Daniel Vetter [Tue, 19 Jul 2016 11:42:54 +0000 (13:42 +0200)]
doc/sphinx: Enable keep_warnings
Unfortunately warnings generated after parsing in sphinx can end up
with entirely bogus files and line numbers as sources. Strangely for
outright errors this is not a problem. Trying to convert warnings to
errors also doesn't fix it.
The only way to get useful output out of sphinx to be able to root
cause the error seems to be enabling keep_warnings, which inserts
a System Message into the actual output. Not pretty at all, but I
don't really want to fix up core rst/sphinx code, and this gets the job
done meanwhile.
Cc: Markus Heiser <markus.heiser@darmarit.de> Cc: Jonathan Corbet <corbet@lwn.net> Cc: linux-doc@vger.kernel.org Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Documentation: add watermark_scale_factor to the list of vm systcl file
Commit 795ae7a0de6b ("mm: scale kswapd watermarks in proportion to
memory") properly added the description of the new knob to
Documentation/sysctl/vm.txt, but forgot to add it to the list of files
in /proc/sys/vm. Let's fix that.
Signed-off-by: Jerome Marchand <jmarchan@redhat.com> Acked-by: Johannes Weiner <hannes@cmpxchg.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Daniel Vetter [Fri, 15 Jul 2016 08:19:30 +0000 (10:19 +0200)]
kernel-doc: Fix up warning output
While trying to make gpu docs warning free I stumbled over one output
which wasn't following proper compiler error output standards. Fix it
up for more quickfix awesomeness.
Cc: Jonathan Corbet <corbet@lwn.net> Cc: Jani Nikula <jani.nikula@intel.com> Cc: linux-doc@vger.kernel.org Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Jonathan Corbet [Mon, 18 Jul 2016 01:24:02 +0000 (19:24 -0600)]
docs: Get rid of some kernel-documentation warnings
Sphinx wants to interpret all literal blocks as being in the chosen
language and complains when an attempt to parse a block fails.
kernel-documentation.rst has a few blocks that are not in C; make that
explicit to shut down the associated warnings.
doc-rst: add an option to ignore DocBooks when generating docs
Sometimes, we want to do a partial build, instead of building
everything. However, right now, if one wants to build just
Sphinx books, it will build also the DocBooks.
Add an option to allow to ignore all DocBooks when building
documentation.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Jani Nikula [Fri, 1 Jul 2016 12:24:44 +0000 (15:24 +0300)]
Documentation/sphinx: skip build if user requested specific DOCBOOKS
If the user requested specific DocBooks to be built using 'make
DOCBOOKS=foo.xml htmldocs', assume no Sphinx build is desired. This
check is transitional, and can be removed once we drop the DocBook
build.
Cc: Markus Heiser <markus.heiser@darmarit.de> Cc: Mauro Carvalho Chehab <mchehab@osg.samsung.com> Fixes: 22cba31bae9d ("Documentation/sphinx: add basic working Sphinx configuration and build") Signed-off-by: Jani Nikula <jani.nikula@intel.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The ``flat-table`` is a double-stage list similar to the ``list-table`` with
some additional features:
* column-span: with the role ``cspan`` a cell can be extended through
additional columns
* row-span: with the role ``rspan`` a cell can be extended through
additional rows
* auto span rightmost cell of a table row over the missing cells on the right
side of that table-row. With Option ``:fill-cells:`` this behavior can
changed from *auto span* to *auto fill*, which automaticly inserts (empty)
list tables
The *list tables* formats are double stage lists. Compared to the
ASCII-art they migth be less comfortable for readers of the
text-files. Their advantage is, that they are easy to create/modify
and that the diff of a modification is much more meaningfull, because
it is limited to the modified content.
The initial implementation was taken from the sphkerneldoc project [1]
Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
[jc: fixed typos and misspellings in the docs] Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Tiezhu Yang [Fri, 17 Jun 2016 01:40:02 +0000 (09:40 +0800)]
Documentation: fix wrong value in md.txt
In the current Documentation/md.txt, the lower limit value of
stripe_cache_size is 16 and the default value is 128, but when
I update kernel to the latest mainline version and RAID5 array
is created by mdadm, then execute the following commands, it
shows an error and a difference respectively.
1) set stripe_cache_size to 16
[root@localhost ~]# echo 16 > /sys/block/md0/md/stripe_cache_size
bash: echo: write error: Invalid argument
2) read the default value of stripe_cache_size
[root@localhost ~]# cat /sys/block/md0/md/stripe_cache_size
256
I read drivers/md/raid5.c and find the following related code:
1) in function 'raid5_set_cache_size':
if (size <= 16 || size > 32768)
return -EINVAL;
2) #define NR_STRIPES 256
So the lower limit value of stripe_cache_size should be 17 and
the default value should be 256.
Signed-off-by: Tiezhu Yang <kernelpatch@126.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Marc MERLIN [Sat, 12 Mar 2016 07:04:19 +0000 (23:04 -0800)]
bcache: documentation updates and corrections
Bcache documentation updates:
- Added new HOWTO/COOKBOOK section
- fixed a few typos
- /sys/block/bcache0/cache_mode is /sys/block/bcache0/bcache/cache_mode
Signed-off-by: Marc MERLIN <marc@merlins.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Jani Nikula [Tue, 7 Jun 2016 09:13:56 +0000 (12:13 +0300)]
Documentation/sphinx: add support for specifying extra export files
Let the user specify file patterns where to look for the EXPORT_SYMBOLs
in addition to the file with kernel-doc comments. This is directly based
on the -export-file FILE option added to kernel-doc in "kernel-doc: add
support for specifying extra files for EXPORT_SYMBOLs", but we extend
that with globbing patterns in the Sphinx extension.
The file patterns are added as options to the :export: and :internal:
arguments of the kernel-doc directive. For example, to extract the
documentation of exported functions from include/net/mac80211.h:
Without the file pattern, no exported functions would be found, as the
EXPORT_SYMBOLs are placed in the various source files under
net/mac80211.
The matched files are also added as dependencies on the document in
Sphinx, as they may affect the output. This is one of the reasons to do
the globbing in the Sphinx extension instead of in scripts/kernel-doc.
The file pattern remains optional, and is not needed if the kernel-doc
comments and EXPORT_SYMBOLs are placed in the source file passed in as
the main argument to the kernel-doc directive. This is the most common
case across the kernel source tree.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Tue, 7 Jun 2016 08:00:52 +0000 (11:00 +0300)]
kernel-doc: add support for specifying extra files for EXPORT_SYMBOLs
If the kernel-doc comments for functions are not in the same file as the
EXPORT_SYMBOL statements, the -export and -internal output selections do
not work as expected. This is typically the case when the kernel-doc
comments are in header files next to the function declarations and the
EXPORT_SYMBOL statements are next to the function definitions in the
source files.
Let the user specify additional source files in which to look for the
EXPORT_SYMBOLs using the new -export-file FILE option, which may be
given multiple times.
The pathological example for this is include/net/mac80211.h, which has
all the kernel-doc documentation for the exported functions defined in a
plethora of source files net/mac80211/*.c.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
kernel-doc: concatenate contents of colliding sections
we started getting (more) errors on duplicate section names, especially
on the default section name "Description":
include/net/mac80211.h:3174: warning: duplicate section name 'Description'
This is usually caused by a slightly unorthodox placement of parameter
descriptions, like in the above case, and kernel-doc resetting back to
the default section more than once within a kernel-doc comment.
Ignore warnings on the duplicate section name automatically assigned by
kernel-doc, and only consider explicitly user assigned duplicate section
names an issue.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jonathan Corbet [Thu, 9 Jun 2016 19:35:05 +0000 (13:35 -0600)]
docs: kernel-doc: Add "example" and "note" to the magic section types
Lots of kerneldoc entries use "example:" or "note:" as section headers.
Until such a time as we can make them use proper markup, make them work as
intended.
Kees Cook [Wed, 18 May 2016 13:37:47 +0000 (06:37 -0700)]
docs: self-protection: rename "leak" to "exposure"
The meaning of "leak" can be both "untracked resource allocation" and
"memory content disclosure". This document's use was entirely of the
latter meaning, so avoid the confusion by using the Common Weakness
Enumeration name for this: Information Exposure (CWE-200). Additionally
adds a section on structure randomization.
Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Jonathan Corbet [Thu, 9 Jun 2016 19:17:11 +0000 (13:17 -0600)]
Merge branch 'sphinx-for-docs-next' into doc/4.8
Jani Nikula says:
Jon, this is v2 of [1] and [2], with a considerable amount of polish and
fixes added. We started dogfooding this within drm-intel, and Daniel has
reviewed the lot and contributed a number of fixes, most notably
accurate file and line number references from Sphinx build
errors/warnings to the kernel-doc comments in source code.
We believe this is now in good shape for merging for v4.8. It's all in
my sphinx-for-docs-next branch that you've already looked at; pull
details below.
When this lands in docs-next and we can backmerge to drm, we'll plunge
ahead and convert gpu.tmpl to rst, and have that ready for v4.8. We
think it's best to contribute that via the drm tree, as it'll involve
splitting up the documentation and likely numerous updates to kernel-doc
comments.
I plan to update Documentation/kernel-doc-nano-HOWTO.txt for Sphinx and
rst, obviously converting it to rst while at it.
Daniel Vetter [Fri, 3 Jun 2016 20:21:35 +0000 (22:21 +0200)]
doc/sphinx: Track line-number of starting blocks
Design is pretty simple: kernel-doc inserts breadcrumbs with line
numbers, and sphinx picks them up. At first I went with a sphinx
comment, but inserting those at random places seriously upsets the
parser, and must be filtered. Hence why this version now uses "#define
LINEO " since one of these ever escape into output it's pretty clear
there is a bug.
It seems to work well, and at least the 2-3 errors where sphinx
complained about something that was not correct in kernel-doc text the
line numbers matched up perfectly.
v2: Instead of noodling around in the parser state machine, create
a ViewList and parse it ourselves. This seems to be the recommended
way, per Jani's suggestion.
v3:
- Split out ViewList pach. Splitting the kernel-doc changes from the
sphinx ones isn't possible, since emitting the LINENO lines wreaks
havoc with the rst formatting. We must filter them.
- Improve the regex per Jani's suggestions, and compile it just once
for speed.
- Now that LINENO lines are eaten, also add them to function parameter
descriptions. Much less content and offset than for in-line struct
member descriptions, but still nice to know which exact continuation
line upsets sphinx.
- Simplify/clarify the line +/-1 business a bit.
v4: Split out the scripts/kernel-doc changes and make line-numbers
opt-in, as suggested by Jani.
Cc: Jani Nikula <jani.nikula@intel.com> Cc: linux-doc@vger.kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Daniel Vetter [Fri, 3 Jun 2016 20:21:34 +0000 (22:21 +0200)]
scripts/kernel-doc: Add option to inject line numbers
Opt-in since this wreaks the rst output and must be removed
by consumers again. This is useful to adjust the linenumbers
for included kernel-doc snippets in shinx. With that sphinx
error message will be accurate when there's issues with the
rst-ness of the kernel-doc comments.
Especially when transitioning a new docbook .tmpl to .rst this
is extremely useful, since you can just use your editors compilation
quickfix list to accurately jump from error to error.
v2:
- Also make sure that we filter the LINENO for purpose/at declaration
start so it only shows for selected blocks, not all of them (Jani).
While at it make it a notch more accurate.
- Avoid undefined $lineno issues. I tried filtering these out at the
callsite, but Jani spotted more when linting the entire kernel.
Unamed unions and similar things aren't stored consistently and end
up with an undefined line number (but also no kernel-doc text, just
the parameter type). Simplify things and filter undefined line
numbers in print_lineno() to catch them all.
v3: Fix LINENO 0 issue for kernel-doc comments without @param: lines
or any other special sections that directly jump to the description
after the "name - purpose" line. Only really possible for functions
without parameters. Noticed by Jani.
Cc: Jani Nikula <jani.nikula@intel.com> Cc: linux-doc@vger.kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Brian Norris [Mon, 23 May 2016 20:37:58 +0000 (13:37 -0700)]
doc: clarify that trace_events= takes a comma-separated list
It took me browsing through the source code to determine that I was,
indeed, using the wrong delimiter in my command lines. So I might as
well document it for the next person.
Signed-off-by: Brian Norris <computersforpeace@gmail.com> Acked-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Mike Danese [Fri, 20 May 2016 04:54:51 +0000 (21:54 -0700)]
mpssd: fix buffer overflow warning
The compilation emits a warning in function ‘snprintf’,
inlined from ‘set_cmdline’ at
../Documentation/mic/mpssd/mpssd.c:1541:9:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:64:10:
warning: call to __builtin___snprintf_chk will always overflow
destination buffer
This was introduced in commit f4a66c204482 ("misc: mic: Update MIC host
daemon with COSM changes") and is fixed by reverting the changes to the
size argument of these snprintf statements.
Cc: Ashutosh Dixit <ashutosh.dixit@intel.com> Signed-off-by: Mike Danese <mikedanese@google.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Daniel Vetter [Wed, 1 Jun 2016 21:46:24 +0000 (23:46 +0200)]
scripts/kernel-doc: Also give functions symbolic names
state3 = prototype parsing, so name them accordingly.
Cc: Jani Nikula <jani.nikula@intel.com> Cc: linux-doc@vger.kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Daniel Vetter [Thu, 2 Jun 2016 12:59:18 +0000 (14:59 +0200)]
doc/sphinx: Stop touching state_machine internals
Instead of just forcefully inserting our kernel-doc input and letting
the state machine stumble over it the recommended way is to create
ViewList, parse that and then return the list of parsed nodes.
Suggested by Jani.
Cc: Jani Nikula <jani.nikula@intel.com> Cc: linux-doc@vger.kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Further up in the state machinery we switch from STATE_NAME to
STATE_DOCBLOCK when we match /$doc_block/. Which means this block of
code here is entirely unreachable, unless there are multiple DOC:
sections within a single kernel-doc comment.
Getting a list of all the files with more than one DOC: section using
$ git grep -c " * DOC:" | grep -v ":1$"
and then doing a full audit of them reveals there are no such comment
blocks in the kernel.
Supporting multiple DOC: sections in a single kernel-doc comment does
not seem like a recommended way of doing things anyway, so nuke the code
for simplicity.
Cc: Jani Nikula <jani.nikula@intel.com> Cc: linux-doc@vger.kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
[Jani: amended the commit message] Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Daniel Vetter [Wed, 1 Jun 2016 09:12:42 +0000 (11:12 +0200)]
doc/sphinx: Pass right filename as source
With this error output becomes almost readable. The line numbers are
still totally bonghits, but that's a lot harder to pull out of
kerneldoc. We'd essentially have to insert some special markers in the
kernel-doc output, split the output along these markers and then
insert each block separately using
Cc: Jani Nikula <jani.nikula@intel.com> Cc: linux-doc@vger.kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Tue, 31 May 2016 15:09:12 +0000 (18:09 +0300)]
Documentation/sphinx: fix kernel-doc extension on python3
Reconcile differences between python2 and python3 on dealing with
stdout, stderr from Popen. This fixes "name 'unicode' is not defined"
errors on python3. We'll need to try to keep the extension working on
both python-sphinx and python3-sphinx so we don't need two copies.
Reported-and-tested-by: Marius Vlad <marius.c.vlad@intel.com> Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Sun, 29 May 2016 06:40:44 +0000 (09:40 +0300)]
kernel-doc: concatenate contents of colliding sections
If there are multiple sections with the same section name, the current
implementation results in several sections by the same heading, with the
content duplicated from the last section to all. Even if there's the
error message, a more graceful approach is to combine all the
identically named sections into one, with concatenated contents.
With the supported sections already limited to select few, there are
massively fewer collisions than there used to be, but this is still
useful for e.g. when function parameters are documented in the middle of
a documentation comment, with description spread out above and
below. (This is not a recommended documentation style, but used in the
kernel nonetheless.)
We can now also demote the error to a warning.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Sun, 29 May 2016 08:35:28 +0000 (11:35 +0300)]
kernel-doc: limit the "section header:" detection to a select few
kernel-doc currently identifies anything matching "section header:"
(specifically a string of word characters and spaces followed by a
colon) as a new section in the documentation comment, and renders the
section header accordingly.
Unfortunately, this turns all uses of colon into sections, mostly
unintentionally. Considering the output, erroneously creating sections
when not intended is always worse than erroneously not creating sections
when intended. For example, a line with "http://example.com" turns into
a "http" heading followed by "//example.com" in normal text style, which
is quite ugly. OTOH, "WARNING: Beware of the Leopard" is just fine even
if "WARNING" does not turn into a heading.
It is virtually impossible to change all the kernel-doc comments, either
way. The compromise is to pick the most commonly used and depended on
section headers (with variants) and accept them as section headers.
The accepted section headers are, case insensitive:
* description:
* context:
* return:
* returns:
Additionally, case sensitive:
* @return:
All of the above are commonly used in the kernel-doc comments, and will
result in worse output if not identified as section headers. Also,
kernel-doc already has some special handling for all of them, so there's
nothing particularly controversial in adding more special treatment for
them.
While at it, improve the whitespace handling surrounding section
names. Do not consider the whitespace as part of the name.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Fri, 27 May 2016 21:41:50 +0000 (00:41 +0300)]
kernel-doc: strip leading whitespace from continued param descs
If a param description spans multiple lines, check any leading
whitespace in the first continuation line, and remove same amount of
whitespace from following lines.
This allows indentation in the multi-line parameter descriptions for
aesthetical reasons while not causing accidentally significant
indentation in the rst output.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Sat, 28 May 2016 11:50:20 +0000 (14:50 +0300)]
kernel-doc: improve handling of whitespace on the first line param description
Handle whitespace on the first line of param text as if it was the empty
string. There is no need to add the newline in this case. This improves
the rst output in particular, where blank lines may be problematic in
parameter lists.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Thu, 26 May 2016 15:30:27 +0000 (18:30 +0300)]
kernel-doc/rst: change the output layout
Move away from field lists, and simply use **strong emphasis** for
section headings on lines of their own. Do not use rst section headings,
because their nesting depth depends on the surrounding context, which
kernel-doc has no knowledge of. Also, they do not need to end up in any
table of contexts or indexes.
There are two related immediate benefits. Field lists are typically
rendered in two columns, while the new style uses the horizontal width
better. With no extra indent on the left, there's no need to be as fussy
about it. Field lists are more susceptible to indentation problems than
the new style.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Thu, 26 May 2016 13:35:02 +0000 (16:35 +0300)]
kernel-doc/rst: add support for struct/union/enum member references
Link "&foo->bar", "&foo->bar()", "&foo.bar", and "&foo.bar()" to the
struct/union/enum foo definition. The members themselves do not
currently have anchors to link to, but this is better than nothing, and
promotes a universal notation.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Thu, 26 May 2016 10:57:18 +0000 (13:57 +0300)]
kernel-doc/rst: add support for &union foo and &typedef foo references
Let the user use "&union foo" and "&typedef foo" to reference foo. The
difference to using "union &foo", "typedef &foo", or just "&foo" (which
are valid too) is that "union" and "typedef" become part of the link
text.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Sun, 29 May 2016 19:27:35 +0000 (22:27 +0300)]
kernel-doc/rst: do not output DOC: section titles for requested ones
If the user requests a specific DOC: section by name, do not output its
section title. In these cases, the surrounding context already has a
heading, and the DOC: section title is only used as an identifier and a
heading for clarity in the source file.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Sun, 29 May 2016 19:19:35 +0000 (22:19 +0300)]
kernel-doc: add names for output selection
Make the output selection a bit more readable by adding constants for
the various types of output selection. While at it, actually call the
variable for choosing what to output $output_selection.
No functional changes.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Thu, 26 May 2016 11:56:05 +0000 (14:56 +0300)]
kernel-doc: add names for states and substates
Make the state machine a bit more readable by adding constants for
parser states and inline member documentation parser substates. While at
it, rename the "split" documentation to "inline" documentation.
No functional changes.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jonathan Corbet [Mon, 7 Mar 2016 03:29:31 +0000 (20:29 -0700)]
sphinx: cheesy script to convert .tmpl files
This script uses pandoc to convert existing DocBook template files to RST
templates. A couple of sed scripts are need to massage things both before
and after the conversion, but the result is then usable with no hand
editing.
[Jani: Change usage to tmplcvt <in> <out>. Fix escaping for docproc
directives. Add support the new kernel-doc extension.]
Signed-off-by: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Sat, 28 May 2016 12:25:41 +0000 (15:25 +0300)]
Documentation/sphinx: set version and release properly
Read the version and release from the top level Makefile (for use when
Sphinx is invoked directly, by e.g. Read the Docs), but override them
via Sphinx command line arguments in a normal documentation build.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Fri, 20 May 2016 08:51:47 +0000 (11:51 +0300)]
Documentation/sphinx: configure the kernel-doc extension
Tell Sphinx where to find the extension, and pass on the kernel src tree
and kernel-doc paths to the extension.
With this, any .rst files under Documentation may contain the kernel-doc
rst directive to include kernel-doc documentation from any source file.
While building, it may be handy to pass kernel-doc extension
configuration on the command line. For example, 'make SPHINXOPTS="-D
kerneldoc_verbosity=0" htmldocs' silences all stderr output from
kernel-doc when the kernel-doc exit code is 0. (The stderr will be
logged unconditionally when the exit code is non-zero.)
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Add an extension to handle kernel-doc directives, to call kernel-doc
according to the arguments and parameters given to the reStructuredText
directive.
The syntax for the kernel-doc directive is:
.. kernel-doc:: FILENAME
:export:
:internal:
:functions: FUNCTION [FUNCTION ...]
:doc: SECTION TITLE
Of the directive options export, internal, functions, and doc, currently
only one option may be given at a time.
The FILENAME is relative from the kernel source tree root.
The extension notifies Sphinx about the document dependency on FILENAME,
causing the document to be rebuilt when the file has been changed.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Thu, 19 May 2016 12:14:05 +0000 (15:14 +0300)]
Documentation/sphinx: add basic working Sphinx configuration and build
Add basic configuration and makefile to build documentation from any
.rst files under Documentation using Sphinx. For starters, there's just
the placeholder index.rst.
At the top level Makefile, hook Sphinx documentation targets alongside
(but independent of) the DocBook toolchain, having both be run on the
various 'make *docs' targets.
All Sphinx processing is placed into Documentation/Makefile.sphinx. Both
that and the Documentation/DocBook/Makefile are now expected to handle
all the documentation targets, explicitly ignoring them if they're not
relevant for that particular toolchain. The changes to the existing
DocBook Makefile are kept minimal.
There is graceful handling of missing Sphinx and rst2pdf (which is
needed for pdf output) by checking for the tool and python module,
respectively, with informative messages to the user.
If the Read the Docs theme (sphinx_rtd_theme) is available, use it, but
otherwise gracefully fall back to the Sphinx default theme, with an
informative message to the user, and slightly less pretty HTML output.
Sphinx can now handle htmldocs, pdfdocs (if rst2pdf is available),
epubdocs and xmldocs targets. The output documents are written into per
output type subdirectories under Documentation/output.
Finally, you can pass options to sphinx-build using the SPHINXBUILD make
variable. For example, 'make SPHINXOPTS=-v htmldocs' for more verbose
output from Sphinx.
This is based on the original work by Jonathan Corbet, but he probably
wouldn't recognize this as his own anymore.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Thu, 21 Jan 2016 11:05:22 +0000 (13:05 +0200)]
kernel-doc: support printing exported and non-exported symbols
Currently we use docproc to figure out which symbols are exported, and
then docproc calls kernel-doc on specific functions, to get
documentation on exported functions. According to git blame and docproc
comments, this is due to historical reasons, as functions and their
corresponding EXPORT_SYMBOL* may have been in different files. However
for more than ten years the recommendation in CodingStyle has been to
place the EXPORT_SYMBOL* immediately after the closing function brace
line.
Additionally, the kernel-doc comments for functions are generally placed
above the function definition in the .c files (i.e. where the
EXPORT_SYMBOL* is) rather than above the declaration in the .h
files. There are some exceptions to this, but AFAICT none of these are
included in DocBook documentation using the "!E" docproc directive.
Therefore, assuming the EXPORT_SYMBOL* and kernel-doc are with the
function definition, kernel-doc can extract the exported vs. not
information by making two passes on the input file. Add support for that
via the new -export and -internal parameters.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Jani Nikula [Thu, 19 May 2016 17:32:48 +0000 (20:32 +0300)]
kernel-doc/rst: fix use of uninitialized value
I'm not quite sure why the errors below are happening, but this fixes
them.
Use of uninitialized value in string ne at ./scripts/kernel-doc line 1819, <IN> line 6494.
Use of uninitialized value $_[0] in join or string at ./scripts/kernel-doc line 1759, <IN> line 6494.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
George Spelvin [Sun, 29 May 2016 05:26:41 +0000 (01:26 -0400)]
Rename other copy of hash_string to hashlen_string
The original name was simply hash_string(), but that conflicted with a
function with that name in drivers/base/power/trace.c, and I decided
that calling it "hashlen_" was better anyway.
But you have to do it in two places.
[ This caused build errors for architectures that don't define
CONFIG_DCACHE_WORD_ACCESS - Linus ]
Mikulas Patocka [Tue, 24 May 2016 20:49:18 +0000 (22:49 +0200)]
hpfs: implement the show_options method
The HPFS filesystem used generic_show_options to produce string that is
displayed in /proc/mounts. However, there is a problem that the options
may disappear after remount. If we mount the filesystem with option1
and then remount it with option2, /proc/mounts should show both option1
and option2, however it only shows option2 because the whole option
string is replaced with replace_mount_options in hpfs_remount_fs.
To fix this bug, implement the hpfs_show_options function that prints
options that are currently selected.
Mikulas Patocka [Tue, 24 May 2016 20:48:33 +0000 (22:48 +0200)]
affs: fix remount failure when there are no options changed
Commit c8f33d0bec99 ("affs: kstrdup() memory handling") checks if the
kstrdup function returns NULL due to out-of-memory condition.
However, if we are remounting a filesystem with no change to
filesystem-specific options, the parameter data is NULL. In this case,
kstrdup returns NULL (because it was passed NULL parameter), although no
out of memory condition exists. The mount syscall then fails with
ENOMEM.
This patch fixes the bug. We fail with ENOMEM only if data is non-NULL.
The patch also changes the call to replace_mount_options - if we didn't
pass any filesystem-specific options, we don't call
replace_mount_options (thus we don't erase existing reported options).
Mikulas Patocka [Tue, 24 May 2016 20:47:00 +0000 (22:47 +0200)]
hpfs: fix remount failure when there are no options changed
Commit ce657611baf9 ("hpfs: kstrdup() out of memory handling") checks if
the kstrdup function returns NULL due to out-of-memory condition.
However, if we are remounting a filesystem with no change to
filesystem-specific options, the parameter data is NULL. In this case,
kstrdup returns NULL (because it was passed NULL parameter), although no
out of memory condition exists. The mount syscall then fails with
ENOMEM.
This patch fixes the bug. We fail with ENOMEM only if data is non-NULL.
The patch also changes the call to replace_mount_options - if we didn't
pass any filesystem-specific options, we don't call
replace_mount_options (thus we don't erase existing reported options).
Fixes: ce657611baf9 ("hpfs: kstrdup() out of memory handling") Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Cc: stable@vger.kernel.org Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
VDSO:
- Build microMIPS VDSO for microMIPS kernels.
- Fix aliasing warning by building with `-fno-strict-aliasing' for
debugging but also tracing them might result in recursion.
Misc:
- Add missing FROZEN hotplug notifier transitions.
- Fix clk binding example for varioius PIC32 devices.
- Fix cpu interrupt controller node-names in the DT files.
- Fix XPA CPU feature separation.
- Fix write_gc0_* macros when writing zero.
- Add inline asm encoding helpers.
- Add missing VZ accessor microMIPS encodings.
- Fix little endian microMIPS MSA encodings.
- Add 64-bit HTW fields and fix its configuration.
- Fix sigreturn via VDSO on microMIPS kernel.
- Lots of typo fixes.
- Add definitions of SegCtl registers and use them"
Linus Torvalds [Sat, 28 May 2016 23:15:25 +0000 (16:15 -0700)]
Merge branch 'hash' of git://ftp.sciencehorizons.net/linux
Pull string hash improvements from George Spelvin:
"This series does several related things:
- Makes the dcache hash (fs/namei.c) useful for general kernel use.
(Thanks to Bruce for noticing the zero-length corner case)
- Converts the string hashes in <linux/sunrpc/svcauth.h> to use the
above.
- Avoids 64-bit multiplies in hash_64() on 32-bit platforms. Two
32-bit multiplies will do well enough.
- Rids the world of the bad hash multipliers in hash_32.
This finishes the job started in commit 689de1d6ca95 ("Minimal
fix-up of bad hashing behavior of hash_64()")
The vast majority of Linux architectures have hardware support for
32x32-bit multiply and so derive no benefit from "simplified"
multipliers.
The few processors that do not (68000, h8/300 and some models of
Microblaze) have arch-specific implementations added. Those
patches are last in the series.
- Overhauls the dcache hash mixing.
The patch in commit 0fed3ac866ea ("namei: Improve hash mixing if
CONFIG_DCACHE_WORD_ACCESS") was an off-the-cuff suggestion.
Replaced with a much more careful design that's simultaneously
faster and better. (My own invention, as there was noting suitable
in the literature I could find. Comments welcome!)
- Modify the hash_name() loop to skip the initial HASH_MIX(). This
would let us salt the hash if we ever wanted to.
- Sort out partial_name_hash().
The hash function is declared as using a long state, even though
it's truncated to 32 bits at the end and the extra internal state
contributes nothing to the result. And some callers do odd things:
- fs/hfs/string.c only allocates 32 bits of state
- fs/hfsplus/unicode.c uses it to hash 16-bit unicode symbols not bytes
- Modify bytemask_from_count to handle inputs of 1..sizeof(long)
rather than 0..sizeof(long)-1. This would simplify users other
than full_name_hash"
Special thanks to Bruce Fields for testing and finding bugs in v1. (I
learned some humbling lessons about "obviously correct" code.)
On the arch-specific front, the m68k assembly has been tested in a
standalone test harness, I've been in contact with the Microblaze
maintainers who mostly don't care, as the hardware multiplier is never
omitted in real-world applications, and I haven't heard anything from
the H8/300 world"
* 'hash' of git://ftp.sciencehorizons.net/linux:
h8300: Add <asm/hash.h>
microblaze: Add <asm/hash.h>
m68k: Add <asm/hash.h>
<linux/hash.h>: Add support for architecture-specific functions
fs/namei.c: Improve dcache hash function
Eliminate bad hash multipliers from hash_32() and hash_64()
Change hash_64() return value to 32 bits
<linux/sunrpc/svcauth.h>: Define hash_str() in terms of hashlen_string()
fs/namei.c: Add hashlen_string() function
Pull out string hash to <linux/stringhash.h>
George Spelvin [Wed, 25 May 2016 18:19:49 +0000 (14:19 -0400)]
h8300: Add <asm/hash.h>
This will improve the performance of hash_32() and hash_64(), but due
to complete lack of multi-bit shift instructions on H8, performance will
still be bad in surrounding code.
Designing H8-specific hash algorithms to work around that is a separate
project. (But if the maintainers would like to get in touch...)
Signed-off-by: George Spelvin <linux@sciencehorizons.net> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Cc: uclinux-h8-devel@lists.sourceforge.jp
George Spelvin [Wed, 25 May 2016 15:06:09 +0000 (11:06 -0400)]
microblaze: Add <asm/hash.h>
Microblaze is an FPGA soft core that can be configured various ways.
If it is configured without a multiplier, the standard __hash_32()
will require a call to __mulsi3, which is a slow software loop.
Instead, use a shift-and-add sequence for the constant multiply.
GCC knows how to do this, but it's not as clever as some.
Signed-off-by: George Spelvin <linux@sciencehorizons.net> Cc: Alistair Francis <alistair.francis@xilinx.com> Cc: Michal Simek <michal.simek@xilinx.com>
George Spelvin [Fri, 27 May 2016 02:11:51 +0000 (22:11 -0400)]
<linux/hash.h>: Add support for architecture-specific functions
This is just the infrastructure; there are no users yet.
This is modelled on CONFIG_ARCH_RANDOM; a CONFIG_ symbol declares
the existence of <asm/hash.h>.
That file may define its own versions of various functions, and define
HAVE_* symbols (no CONFIG_ prefix!) to suppress the generic ones.
Included is a self-test (in lib/test_hash.c) that verifies the basics.
It is NOT in general required that the arch-specific functions compute
the same thing as the generic, but if a HAVE_* symbol is defined with
the value 1, then equality is tested.
Signed-off-by: George Spelvin <linux@sciencehorizons.net> Cc: Geert Uytterhoeven <geert@linux-m68k.org> Cc: Greg Ungerer <gerg@linux-m68k.org> Cc: Andreas Schwab <schwab@linux-m68k.org> Cc: Philippe De Muyter <phdm@macq.eu> Cc: linux-m68k@lists.linux-m68k.org Cc: Alistair Francis <alistai@xilinx.com> Cc: Michal Simek <michal.simek@xilinx.com> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Cc: uclinux-h8-devel@lists.sourceforge.jp
George Spelvin [Mon, 23 May 2016 11:43:58 +0000 (07:43 -0400)]
fs/namei.c: Improve dcache hash function
Patch 0fed3ac866 improved the hash mixing, but the function is slower
than necessary; there's a 7-instruction dependency chain (10 on x86)
each loop iteration.
Word-at-a-time access is a very tight loop (which is good, because
link_path_walk() is one of the hottest code paths in the entire kernel),
and the hash mixing function must not have a longer latency to avoid
slowing it down.
There do not appear to be any published fast hash functions that:
1) Operate on the input a word at a time, and
2) Don't need to know the length of the input beforehand, and
3) Have a single iterated mixing function, not needing conditional
branches or unrolling to distinguish different loop iterations.
One of the algorithms which comes closest is Yann Collet's xxHash, but
that's two dependent multiplies per word, which is too much.
The key insights in this design are:
1) Barring expensive ops like multiplies, to diffuse one input bit
across 64 bits of hash state takes at least log2(64) = 6 sequentially
dependent instructions. That is more cycles than we'd like.
2) An operation like "hash ^= hash << 13" requires a second temporary
register anyway, and on a 2-operand machine like x86, it's three
instructions.
3) A better use of a second register is to hold a two-word hash state.
With careful design, no temporaries are needed at all, so it doesn't
increase register pressure. And this gets rid of register copying
on 2-operand machines, so the code is smaller and faster.
4) Using two words of state weakens the requirement for one-round mixing;
we now have two rounds of mixing before cancellation is possible.
5) A two-word hash state also allows operations on both halves to be
done in parallel, so on a superscalar processor we get more mixing
in fewer cycles.
I ended up using a mixing function inspired by the ChaCha and Speck
round functions. It is 6 simple instructions and 3 cycles per iteration
(assuming multiply by 9 can be done by an "lea" instruction):
x ^= *input++;
y ^= x; x = ROL(x, K1);
x += y; y = ROL(y, K2);
y *= 9;
Not only is this reversible, two consecutive rounds are reversible:
if you are given the initial and final states, but not the intermediate
state, it is possible to compute both input words. This means that at
least 3 words of input are required to create a collision.
(It also has the property, used by hash_name() to avoid a branch, that
it hashes all-zero to all-zero.)
The rotate constants K1 and K2 were found by experiment. The search took
a sample of random initial states (I used 1023) and considered the effect
of flipping each of the 64 input bits on each of the 128 output bits two
rounds later. Each of the 8192 pairs can be considered a biased coin, and
adding up the Shannon entropy of all of them produces a score.
The best-scoring shifts also did well in other tests (flipping bits in y,
trying 3 or 4 rounds of mixing, flipping all 64*63/2 pairs of input bits),
so the choice was made with the additional constraint that the sum of the
shifts is odd and not too close to the word size.
The final state is then folded into a 32-bit hash value by a less carefully
optimized multiply-based scheme. This also has to be fast, as pathname
components tend to be short (the most common case is one iteration!), but
there's some room for latency, as there is a fair bit of intervening logic
before the hash value is used for anything.
(Performance verified with "bonnie++ -s 0 -n 1536:-2" on tmpfs. I need
a better benchmark; the numbers seem to show a slight dip in performance
between 4.6.0 and this patch, but they're too noisy to quote.)
Special thanks to Bruce fields for diligent testing which uncovered a
nasty fencepost error in an earlier version of this patch.
[checkpatch.pl formatting complaints noted and respectfully disagreed with.]
Signed-off-by: George Spelvin <linux@sciencehorizons.net> Tested-by: J. Bruce Fields <bfields@redhat.com>
George Spelvin [Fri, 27 May 2016 03:00:23 +0000 (23:00 -0400)]
Eliminate bad hash multipliers from hash_32() and hash_64()
The "simplified" prime multipliers made very bad hash functions, so get rid
of them. This completes the work of 689de1d6ca.
To avoid the inefficiency which was the motivation for the "simplified"
multipliers, hash_64() on 32-bit systems is changed to use a different
algorithm. It makes two calls to hash_32() instead.
drivers/media/usb/dvb-usb-v2/af9015.c uses the old GOLDEN_RATIO_PRIME_32
for some horrible reason, so it inherits a copy of the old definition.
Signed-off-by: George Spelvin <linux@sciencehorizons.net> Cc: Antti Palosaari <crope@iki.fi> Cc: Mauro Carvalho Chehab <m.chehab@samsung.com>
George Spelvin [Fri, 27 May 2016 02:22:01 +0000 (22:22 -0400)]
Change hash_64() return value to 32 bits
That's all that's ever asked for, and it makes the return
type of hash_long() consistent.
It also allows (upcoming patch) an optimized implementation
of hash_64 on 32-bit machines.
I tried adding a BUILD_BUG_ON to ensure the number of bits requested
was never more than 32 (most callers use a compile-time constant), but
adding <linux/bug.h> to <linux/hash.h> breaks the tools/perf compiler
unless tools/perf/MANIFEST is updated, and understanding that code base
well enough to update it is too much trouble. I did the rest of an
allyesconfig build with such a check, and nothing tripped.
Signed-off-by: George Spelvin <linux@sciencehorizons.net>
George Spelvin [Fri, 20 May 2016 17:31:33 +0000 (13:31 -0400)]
<linux/sunrpc/svcauth.h>: Define hash_str() in terms of hashlen_string()
Finally, the first use of previous two patches: eliminate the
separate ad-hoc string hash functions in the sunrpc code.
Now hash_str() is a wrapper around hash_string(), and hash_mem() is
likewise a wrapper around full_name_hash().
Note that sunrpc code *does* call hash_mem() with a zero length, which
is why the previous patch needed to handle that in full_name_hash().
(Thanks, Bruce, for finding that!)
This also eliminates the only caller of hash_long which asks for
more than 32 bits of output.
The comment about the quality of hashlen_string() and full_name_hash()
is jumping the gun by a few patches; they aren't very impressive now,
but will be improved greatly later in the series.
Signed-off-by: George Spelvin <linux@sciencehorizons.net> Tested-by: J. Bruce Fields <bfields@redhat.com> Acked-by: J. Bruce Fields <bfields@redhat.com> Cc: Jeff Layton <jlayton@poochiereds.net> Cc: linux-nfs@vger.kernel.org
George Spelvin [Fri, 20 May 2016 12:41:37 +0000 (08:41 -0400)]
fs/namei.c: Add hashlen_string() function
We'd like to make more use of the highly-optimized dcache hash functions
throughout the kernel, rather than have every subsystem create its own,
and a function that hashes basic null-terminated strings is required
for that.
(The name is to emphasize that it returns both hash and length.)
It's actually useful in the dcache itself, specifically d_alloc_name().
Other uses in the next patch.
full_name_hash() is also tweaked to make it more generally useful:
1) Take a "char *" rather than "unsigned char *" argument, to
be consistent with hash_name().
2) Handle zero-length inputs. If we want more callers, we don't want
to make them worry about corner cases.
Signed-off-by: George Spelvin <linux@sciencehorizons.net>
Linus Torvalds [Sat, 28 May 2016 19:38:50 +0000 (12:38 -0700)]
Merge branch 'i2c/for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux
Pull i2c fix from Wolfram Sang:
"A fix for a regression introduced yesterday.
The regression didn't show up here locally because I did not have
PAGE_POISONING enabled. And buildbots discovered this only after it
hit your tree. Thanks to Dan for the quick response"
* 'i2c/for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux:
i2c: dev: use after free in detach