Lines Matching +full:in +full:- +full:kernel
1 .. SPDX-License-Identifier: (GPL-2.0+ OR CC-BY-4.0)
3 If you want to distribute this text under CC-BY-4.0 only, please use 'The
4 Linux kernel developers' for author attribution and link this as source:
5 …https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/Documentation/admin-guide…
7 Note: Only the content of this RST file as found in the Linux kernel sources
8 is available under CC-BY-4.0, as versions of this text that were processed
9 (for example by the kernel's build system) might contain content taken from
22 <https://lore.kernel.org/lkml/>`_ and the `Linux stable mailing list
23 <https://lore.kernel.org/stable/>`_ archives for matching reports to join. If
25 <https://kernel.org/>`_. If it still shows the issue, report it to the stable
26 mailing list (stable@vger.kernel.org) and CC the regressions list
28 list for the subsystem in question.
30 In all other cases try your best guess which kernel part might be causing the
33 mailing list in CC. Check the destination's archives for matching reports;
34 search the `LKML <https://lore.kernel.org/lkml/>`_ and the web, too. If you
35 don't find any to join, install `the latest mainline kernel
36 <https://kernel.org/>`_. If the issue is present there, send a report.
38 The issue was fixed there, but you would like to see it resolved in a still
40 If it shows the problem, search for the change that fixed it in mainline and
41 check if backporting is in the works or was discarded; if it's neither, ask
44 **General remarks**: When installing and testing a kernel as outlined above,
45 ensure it's vanilla (IOW: not patched and not using add-on modules). Also make
46 sure it's built and running in a healthy environment and not already tainted
49 If you are facing multiple issues with the Linux kernel at once, report each
51 issue, like the kernel and the distro used. In case of a regression, CC the
53 to pin-point the culprit with a bisection; if you succeed, include its
54 commit-id and CC everyone in the sign-off-by chain.
60 Step-by-step guide how to report issues to the kernel maintainers
63 The above TL;DR outlines roughly how to report issues to the Linux kernel
67 step-by-step approach. It still tries to be brief for readability and leaves
68 out a lot of details; those are described below the step-by-step guide in a
69 reference section, which explains each of the steps in more detail.
71 Note: this section covers a few more aspects than the TL;DR and does things in
72 a slightly different order. That's in your interest, to make sure you notice
73 early if an issue that looks like a Linux kernel problem is actually caused by
74 something else. These steps thus help to ensure the time you invest in this
75 process won't feel wasted in the end:
77 * Are you facing an issue with a Linux kernel a hardware or software vendor
78 provided? Then in almost all cases you are better off to stop reading this
84 search engine; additionally, check the archives of the `Linux Kernel Mailing
85 List (LKML) <https://lore.kernel.org/lkml/>`_. If you find matching reports,
90 need special handling in some steps that are about to follow.
92 * Make sure it's not the kernel's surroundings that are causing the issue
98 kernel modules on-the-fly, which solutions like DKMS might be doing locally
101 * Check if your kernel was 'tainted' when the issue occurred, as the event
102 that made the kernel set this flag might be causing the issue you face.
107 needs to get reported to the kernel developers separately, unless they are
112 'Dealing with regressions within a stable and longterm kernel line'.
114 * Locate the driver or kernel subsystem that seems to be causing the issue.
116 time this won't be bugzilla.kernel.org, as issues typically need to be sent
119 * Search the archives of the bug tracker or mailing list in question
125 * Unless you are already running the latest 'mainline' Linux kernel, better
127 the latest 'stable' Linux can be an acceptable alternative in some
129 approach, but in that development phase it can be an even better idea to
134 * Ensure the kernel you just installed does not 'taint' itself when
137 * Reproduce the issue with the kernel you just installed. If it doesn't show
144 that hear about it for the first time. And if you learned something in this
148 decoding the kernel log to find the line of code that triggered the error.
154 issue. Always mention a few things: the latest kernel version you installed
156 reproduce the issue. Ideally, make the kernel's build configuration
166 special care which is explained in 'Special handling for high priority
170 outcome in one way or the other. Thus react publicly and in a timely manner
177 Reporting regressions within a stable and longterm kernel line
178 --------------------------------------------------------------
181 the point about regression within a stable or longterm kernel version line. You
187 * Check if the kernel developers still maintain the Linux kernel version
188 line you care about: go to the `front page of kernel.org
189 <https://kernel.org/>`_ and make sure it mentions
193 <https://lore.kernel.org/stable/>`_ for existing reports.
196 kernel. Ensure this kernel is not tainted and still shows the problem, as
198 problem with a vendor kernel, check a vanilla build of the last version
202 (stable@vger.kernel.org) and CC the Linux regressions mailing list
203 (regressions@lists.linux.dev); if you suspect the cause in a particular
209 The reference section below explains each of these steps in more detail.
212 Reporting issues only occurring in older kernel version lines
213 -------------------------------------------------------------
215 This subsection is for you, if you tried the latest mainline kernel as outlined
217 see the issue fixed in a still supported stable or longterm series or vendor
221 might not get the issue solved in older releases: the fix might be too big
224 * Perform the first three steps in the section "Dealing with regressions
225 within a stable and longterm kernel line" above.
227 * Search the Linux kernel version control system for the change that fixed
228 the issue in mainline, as its commit message might tell you if the fix is
231 or peer-review possible fixes; then check the discussions if the fix was
233 all, join the newest discussion, asking if it's in the cards.
240 The reference section below explains each of these steps in more detail.
243 Reference section: Reporting issues to the kernel maintainers
246 The detailed guides above outline all the major steps in brief fashion, which
256 * The Linux kernel developers are well aware this process is complicated and
258 that would require work in various places as well as some infrastructure,
263 request fixes from developers in the upstream Linux kernel community: such
264 contracts are completely outside the scope of the Linux kernel, its
266 anything such a contract guarantees in this context, not even if the
267 developer handling the issue works for the vendor in question. If you want
269 so, you might want to mention you'd like to see the issue fixed in the
270 upstream Linux kernel; motivate them by saying it's the only way to ensure
271 the fix in the end will get incorporated in all Linux distributions.
277 <http://www.catb.org/esr/faqs/smart-questions.html>`_, and `How to ask good
278 questions <https://jvns.ca/blog/good-questions/>`_.
281 issues to the Linux kernel developers.
284 Make sure you're using the upstream Linux kernel
285 ------------------------------------------------
287 *Are you facing an issue with a Linux kernel a hardware or software vendor
288 provided? Then in almost all cases you are better off to stop reading this
293 Like most programmers, Linux kernel developers don't like to spend time dealing
296 easily happen when it comes to the kernel and often leads to frustration on both
297 sides. That's because almost all Linux-based kernels pre-installed on devices
299 distributors are quite distant from the official Linux kernel as distributed by
300 kernel.org: these kernels from these vendors are often ancient from the point of
304 Linux kernel developers: an issue you face with one of them might have been
305 fixed by the Linux kernel developers months or years ago already; additionally,
309 report and, in case it turns out to be an upstream issue, fix it directly
310 upstream or forward the report there. In practice that often does not work out
312 vendor by installing the very latest Linux kernel core yourself. If that's an
313 option for you move ahead in this process, as a later step in this guide will
317 developers in fact are willing to handle reports about issues occurring with
318 vendor kernels. If they do in the end highly depends on the developers and the
319 issue in question. Your chances are quite good if the distributor applied only
320 small modifications to a kernel based on a recent Linux version; that for
323 with kernels from distributions shipping the latest stable kernel, as long as
325 regular Fedora releases, and openSUSE Tumbleweed. But keep in mind, you better
326 want to use a mainline Linux and avoid using a stable kernel for this
327 process, as outlined in the section 'Install a fresh kernel for testing' in more
331 or heavily modified vendor kernel to the upstream Linux developers. But note,
338 --------------------------------------
341 search engine; additionally, check the archives of the Linux Kernel Mailing
346 time for everyone involved, especially you as the reporter. So it's in your own
354 search the `Linux Kernel Mailing List (LKML) archives
355 <https://lore.kernel.org/lkml/>`_.
363 the name of the kernel driver or the name of the affected hardware component.
369 In case you find an existing report about your issue, join the discussion, as
371 important even when a fix is prepared or in its final stages already, as
376 Note, searching `bugzilla.kernel.org <https://bugzilla.kernel.org/>`_ might also
378 reports. If you find the latter, just keep in mind: most subsystems expect
379 reports in different places, as described below in the section "Check where you
382 the issue already got reported as outlined in this document and if not consider
387 -----------------------
391 need special handling in some steps that are about to follow.*
393 Linus Torvalds and the leading Linux kernel developers want to see some issues
395 handled slightly differently in the reporting process. Three type of cases
399 the Linux kernel does not work with a newer one or somehow works worse with it.
402 an application shows erratic behavior with a newer kernel, which might happen
403 due to incompatible changes in the interface between the kernel and the
405 increased power consumption also qualify as regression. But keep in mind: the
406 new kernel needs to be built with a configuration that is similar to the one
407 from the old kernel (see below how to achieve that). That's because the kernel
413 'Documentation/admin-guide/security-bugs.rst' before proceeding, as it
417 happens. That's for example the case when a Linux kernel corrupts the data it's
419 issue when the kernel suddenly stops working with an error message ('kernel
421 fatal error where the kernel stop itself) with a 'Oops' (a recoverable error),
422 as the kernel remains running after the latter.
426 ----------------------------
428 *Make sure it's not the kernel's surroundings that are causing the issue
431 Problems that look a lot like a kernel issue are sometimes caused by build or
435 * Use proven tools when building your kernel, as bugs in the compiler or the
436 binutils can cause the resulting kernel to misbehave.
441 potential kernel issue.
444 main memory for example can result in a multitude of issues that will
445 manifest itself in problems looking like kernel issues.
448 system in question with ``fsck``, as it might be damaged in a way that leads
449 to unexpected kernel behavior.
452 changed in parallel to updating the kernel. The problem for example might be
455 into a new kernel for the first time. Updating the systems BIOS or changing
456 something in the BIOS Setup can also lead to problems that on look a lot
457 like a kernel regression.
461 -----------------------
466 especially if you fiddle with crucial parts like the kernel of its operating
467 system. That's what you are about to do in this process. Thus, make sure to
473 Make sure your kernel doesn't get enhanced
474 ------------------------------------------
477 kernel modules on-the-fly, which solutions like DKMS might be doing locally
481 your kernel gets enhanced in any way. That's why you should remove or disable
482 mechanisms like akmods and DKMS: those build add-on kernel modules
483 automatically, for example when you install a new Linux kernel or boot it for
490 module not part of the Linux kernel. That why your might need to uninstall the
491 packages with such software to get rid of any 3rd party kernel module.
495 ------------------
497 *Check if your kernel was 'tainted' when the issue occurred, as the event
498 that made the kernel set this flag might be causing the issue you face.*
500 The kernel marks itself with a 'taint' flag when something happens that might
501 lead to follow-up errors that look totally unrelated. The issue you face might
502 be such an error if your kernel is tainted. That's why it's in your interest to
505 install the latest mainline kernel; you will need to check the taint flag again
506 then, as that's when it matters because it's the kernel the report will focus
509 On a running system is easy to check if the kernel tainted itself: if ``cat
510 /proc/sys/kernel/tainted`` returns '0' then the kernel is not tainted and
511 everything is fine. Checking that file is impossible in some situations; that's
512 why the kernel also mentions the taint status when it reports an internal
513 problem (a 'kernel bug'), a recoverable error (a 'kernel Oops') or a
514 non-recoverable error before halting operation (a 'kernel panic'). Look near
516 line starting with 'CPU:'. It should end with 'Not tainted' if the kernel was
520 If your kernel is tainted, study 'Documentation/admin-guide/tainted-kernels.rst'
524 1. A recoverable error (a 'kernel Oops') occurred and the kernel tainted
525 itself, as the kernel knows it might misbehave in strange ways after that
526 point. In that case check your kernel or system log and look for a section
531 That's the first Oops since boot-up, as the '#1' between the brackets shows.
533 follow-up problem to that first Oops, even if both look totally unrelated.
538 the cause for the Oops might already be fixed in the newer Linux kernel
539 version you are going to install later in this process.
541 2. Your system uses a software that installs its own kernel modules, for
542 example Nvidia's proprietary graphics driver or VirtualBox. The kernel
544 they are Open Source): they sometimes cause errors in unrelated kernel
547 Linux kernel developers. Most of the time the easiest way to do that is:
551 3. The kernel also taints itself when it's loading a module that resides in
552 the staging tree of the Linux kernel source. That's a special area for
553 code (mostly drivers) that does not yet fulfill the normal Linux kernel
555 obviously okay if the kernel is tainted; just make sure the module in
556 question is the only reason for the taint. If the issue happens in an
558 by specifying ``foo.blacklist=1`` as kernel parameter (replace 'foo' with
559 the name of the module in question).
563 -------------------------------
568 needs to get reported to the kernel developers separately, unless they are
573 various issues in one report also makes it quite difficult for others to tear
574 it apart. Hence, only combine issues in one report if they are very strongly
578 happens with other kernel versions. Therefore, it will make your work easier if
584 to ignore this advice if you are experienced enough to tell a one-time error
585 due to faulty hardware apart from a kernel issue that rarely happens and thus
589 Regression in stable or longterm kernel?
590 ----------------------------------------
594 'Dealing with regressions within a stable and longterm kernel line'.*
596 Regression within a stable and longterm kernel version line are something the
598 regression in the main development branch, as they can quickly affect a lot of
601 regressions with newer kernel version line (say something broke when switching
606 -----------------------------------------
608 *Locate the driver or kernel subsystem that seems to be causing the issue.
610 time this won't be bugzilla.kernel.org, as issues typically need to be sent
613 It's crucial to send your report to the right people, as the Linux kernel is a
620 Problem is: the Linux kernel lacks a central bug tracker where you can simply
624 kernel developers and experts. For everybody else the MAINTAINERS file is the
630 the WiFi in your Laptop suddenly misbehaves after updating the kernel. In that
631 case it's likely an issue in the WiFi driver. Obviously it could also be some
639 In case of a problem with the WiFi driver you for example might want to look at
640 the output of ``lspci -k``, as it lists devices on the PCI/PCIe bus and the
641 kernel module driving it::
643 [user@something ~]$ lspci -k
647 Kernel driver in use: ath10k_pci
648 Kernel modules: ath10k_pci
652 other internal bus. In those cases you might want to check your WiFi manager or
657 …[user@something ~]$ realpath --relative-to=/sys/module/ /sys/class/net/wlp58s0/device/driver/module
660 In case tricks like these don't bring you any further, try to search the
661 internet on how to narrow down the driver or subsystem in question. And if you
665 Once you know the driver or subsystem, you want to search for it in the
666 MAINTAINERS file. In the case of 'ath10k_pci' you won't find anything, as the
675 Web-page: https://wireless.wiki.kernel.org/en/users/Drivers/ath10k
676 SCM: git git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git
680 MAINTAINERS file found in the root of the Linux source tree. 'Mail:' for
695 Linux kernel development is completely driven by mail. Very few subsystems use
696 a bug tracker, and only some of those rely on bugzilla.kernel.org.
698 In this and many other cases you thus have to look for lines starting with
703 issue reports sent by email, make sure to add the Linux Kernel Mailing List
704 (LKML) <linux-kernel@vger.kernel.org> to CC. Don't omit either of the mailing
716 called with a path to the source code in question. For drivers compiled as
719 …$ modinfo ath10k_pci | grep filename | sed 's!/lib/modules/.*/kernel/!!; s!filename:!!; s!\.ko\(\|…
724 $ ./scripts/get_maintainer.pl -f drivers/net/wireless/ath/ath10k*
728 linux-wireless@vger.kernel.org (open list:NETWORKING DRIVERS (WIRELESS))
729 netdev@vger.kernel.org (open list:NETWORKING DRIVERS)
730 linux-kernel@vger.kernel.org (open list)
734 the code as well as the Linux Kernel Mailing List (LKML). In this case you thus
736 'ath10k@lists.infradead.org' and 'linux-kernel@vger.kernel.org' in CC.
738 Note: in case you cloned the Linux sources with git you might want to call
739 ``get_maintainer.pl`` a second time with ``--git``. The script then will look
740 at the commit history to find which people recently worked on the code in
742 can easily send you in a wrong direction. That for example happens quickly in
744 modified during tree-wide cleanups by developers that do not care about the
749 ---------------------------------------
751 *Search the archives of the bug tracker or mailing list in question
759 often find its archives on `lore.kernel.org <https://lore.kernel.org/>`_.
761 But some list are hosted in different places. That for example is the case for
762 the ath10k WiFi driver used as example in the previous step. But you'll often
768 quite a few other lists miss a way to search the archives. In those cases use a
773 It's also wise to check the internet, LKML and maybe bugzilla.kernel.org again
774 at this point. If your report needs to be filed in a bug tracker, you may want
785 Install a fresh kernel for testing
786 ----------------------------------
788 *Unless you are already running the latest 'mainline' Linux kernel, better
790 the latest 'stable' Linux can be an acceptable alternative in some
792 approach, but in that development phase it can be an even better idea to
797 As mentioned in the detailed explanation for the first step already: Like most
798 programmers, Linux kernel developers don't like to spend time dealing with
800 waste everybody's time, especially yours. That's why it's in everybody's
806 In the scope of the kernel "latest upstream" normally means:
808 * Install a mainline kernel; the latest stable kernel can be an option, but
811 explains all of this in more detail.
813 * The over next subsection describes way to obtain and install such a kernel.
814 It also outlines that using a pre-compiled kernel are fine, but better are
816 kernel.org <https://kernel.org/>`_ and not modified or enhanced in any way.
821 Head over to `kernel.org <https://kernel.org/>`_ to find out which version you
824 mainline, which most of the time will point to a pre-release with a version
825 number like '5.8-rc2'. If that's the case, you'll want to use this mainline
826 kernel for testing, as that where all fixes have to be applied first. Do not let
830 In about two out of every nine to ten weeks, mainline might point you to a
832 suspending the reporting process until the first pre-release of the next
833 version (5.8-rc1) shows up on kernel.org. That's because the Linux development
834 cycle then is in its two-week long 'merge window'. The bulk of the changes and
836 more risky to use mainline during this period. Kernel developers are also often
840 a newer kernel version anyway, as outlined below in the section 'Duties after
844 to that if you're dealing with something that shouldn't wait. In that case
845 consider obtaining the latest mainline kernel via git (see below) or use the
846 latest stable version offered on kernel.org. Using that is also acceptable in
847 case mainline for some reason does currently not work for you. An in general:
851 Better avoid using the latest stable kernel outside merge windows, as all fixes
853 kernel is so important: any issue you want to see fixed in older version lines
854 needs to be fixed in mainline first before it can get backported, which can
863 it fixed in older version lines, if that's in the cards for the fix in question.
865 How to obtain a fresh Linux kernel
868 **Using a pre-compiled kernel**: This is often the quickest, easiest, and safest
869 way for testing — especially is you are unfamiliar with the Linux kernel. The
870 problem: most of those shipped by distributors or add-on repositories are build
875 But you are in luck if you are using a popular Linux distribution: for quite a
877 latest mainline or stable Linux built as vanilla kernel. It's totally okay to
880 versions as offered on kernel.org. The packages are likely unsuitable if they
884 Please note that you might need to build your own kernel manually later: that's
885 sometimes needed for debugging or testing fixes, as described later in this
886 document. Also be aware that pre-compiled kernels might lack debug symbols that
887 are needed to decode messages the kernel prints when a panic, Oops, warning, or
889 kernel yourself (see the end of this subsection and the section titled 'Decode
893 often best served by obtaining the latest Linux kernel sources straight from the
894 `official development repository on kernel.org
895 <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/>`_.
896 Those are likely a bit ahead of the latest mainline pre-release. Don't worry
897 about it: they are as reliable as a proper pre-release, unless the kernel's
898 development cycle is currently in the middle of a merge window. But even then
902 downloading the sources as tarball from `kernel.org <https://kernel.org/>`_.
904 How to actually build a kernel is not described here, as many websites explain
906 those how-to's that suggest to use ``make localmodconfig``, as that tries to
907 pick up the configuration of your current kernel and then tries to adjust it
908 somewhat for your system. That does not make the resulting kernel any better,
911 Note: If you are dealing with a panic, Oops, warning, or BUG from the kernel,
912 please try to enable CONFIG_KALLSYMS when configuring your kernel.
916 build a kernel by quite a bit. But that's worth it, as these options will allow
918 section 'Decode failure messages' below explains this in more detail.
920 But keep in mind: Always keep a record of the issue encountered in case it is
926 ------------------
928 *Ensure the kernel you just installed does not 'taint' itself when
931 As outlined above in more detail already: the kernel sets a 'taint' flag when
932 something happens that can lead to follow-up errors that look totally
933 unrelated. That's why you need to check if the kernel you just installed does
934 not set this flag. And if it does, you in almost all the cases needs to
939 Reproduce issue with the fresh kernel
940 -------------------------------------
942 *Reproduce the issue with the kernel you just installed. If it doesn't show
946 Check if the issue occurs with the fresh Linux kernel version you just
948 line and abandoning your plan to report the issue. But keep in mind that other
949 users might still be plagued by it, as long as it's not fixed in either stable
950 and longterm version from kernel.org (and thus vendor kernels derived from
952 head over to the section "Details about reporting issues only occurring in
953 older kernel version lines" below.
957 ---------------------------------------
962 that hear about it for the first time. And if you learned something in this
967 thus easy to understand in written form. Include all important details, but at
970 In this in the previous steps you likely have learned a thing or two about the
976 -----------------------
979 decoding the kernel log to find the line of code that triggered the error.*
981 When the kernel detects an internal problem, it will log some information about
982 the executed code. This makes it possible to pinpoint the exact line in the
985 your kernel. If you did so, consider to decode the information from the
986 kernel's log. That will make it a lot easier to understand what lead to the
990 Decoding can be done with a script you find in the Linux source tree. If you
991 are running a kernel you compiled yourself earlier, call it like this::
993 …[user@something ~]$ sudo dmesg | ./linux-5.10.5/scripts/decode_stacktrace.sh ./linux-5.10.5/vmlinux
995 If you are running a packaged vanilla kernel, you will likely have to install
1000 [user@something ~]$ sudo dmesg | ./linux-5.10.5/scripts/decode_stacktrace.sh \
1001 /usr/lib/debug/lib/modules/5.10.10-4.1.x86_64/vmlinux /usr/src/kernels/5.10.10-4.1.x86_64/
1004 the code the kernel was executing when the error occurred::
1010 …[ 68.387301] RIP: 0010:test_module_init (/home/username/linux-5.10.5/test-module/test-module.c:1…
1012 In this case the executed code was built from the file
1013 '~/linux-5.10.5/test-module/test-module.c' and the error occurred by the
1014 instructions found in line '16'.
1016 The script will similarly decode the addresses mentioned in the section
1019 the code section the kernel was executing.
1022 reason for it in the report. If you're lucky, it might not be needed. And if it
1024 of several ways to decode kernel stack traces. Sometimes different steps will
1026 needed in your case, developers will tell you what to do.
1030 ----------------------------
1035 Linux lead developer Linus Torvalds insists that the Linux kernel never
1046 'Documentation/admin-guide/bug-bisect.rst' describes in detail. That process
1047 will often require you to build about ten to twenty kernel images, trying to
1050 Thanks to a 'binary search' this will lead you to the one commit in the source
1056 Note, a bisection needs a bit of know-how, which not everyone has, and quite a
1059 don't want to go down that route at least find out which mainline kernel
1061 5.5.15 to 5.8.4, then try at least all the mainline releases in that area (5.6,
1063 regression in a stable or longterm kernel, avoid testing versions which number
1066 version which introduced the regression, feel free to move on in the reporting
1067 process. But keep in mind: it depends on the issue at hand if the developers
1073 the kernel and not by something else, as outlined above already.
1075 In the whole process keep in mind: an issue only qualifies as regression if the
1076 older and the newer kernel got built with a similar configuration. The best way
1078 kernel freshly to each newer kernel version you try. Afterwards run ``make
1083 -------------------------
1086 issue. Always mention a few things: the latest kernel version you installed
1088 reproduce the issue. Ideally, make the kernel's build configuration
1098 special care which is explained in 'Special handling for high priority
1102 that is partly explained by the three documents linked to in the preface above.
1104 things specific to the Linux kernel.
1112 and write the detailed report first. ;-)
1117 Describe in detail how your issue happens with the fresh vanilla kernel you
1118 installed. Try to include the step-by-step instructions you wrote and optimized
1119 earlier that outline how you and ideally others can reproduce the issue; in
1127 * the output from ``cat /proc/version``, which contains the Linux kernel
1133 * the architecture of the CPU and the operating system (``uname -mi``)
1136 subject and the commit-id of the change that is causing it.
1138 In a lot of cases it's also wise to make two more things available to those
1141 * the configuration used for building your Linux kernel (the '.config' file)
1143 * the kernel's messages that you get from ``dmesg`` written to a file. Make
1144 sure that it starts with a line like 'Linux version 5.8-1
1147 boot phase already got discarded. In this case instead consider using
1148 ``journalctl -b 0 -k``; alternatively you can also reboot, reproduce the
1152 your report. If you are filing the issue in a bug tracker then attach them to
1157 service, a ticket created just for this purpose on `bugzilla.kernel.org
1158 <https://bugzilla.kernel.org/>`_, ...) and include a link to them in your
1164 * Put the files aside and mention you will send them later in individual
1166 went out. ;-)
1174 * If you are dealing with a 'warning', an 'OOPS' or a 'panic' from the kernel,
1184 nothing in common. Hence, in such cases add the exact model number, which
1190 * Mention the relevant software in use. If you have problems with loading
1191 modules, you want to mention the versions of kmod, systemd, and udev in use.
1193 libdrm and Mesa; also specify your Wayland compositor or the X-Server and
1195 corresponding filesystem utilities (e2fsprogs, btrfs-progs, xfsprogs, ...).
1197 * Gather additional information from the kernel that might be of interest. The
1198 output from ``lspci -nn`` will for example help others to identify what
1200 make the output from ``sudo lspci -vvv`` available, as that provides
1205 information. One such tool is ``alsa-info.sh`` `which the audio/sound
1206 subsystem developers provide <https://www.alsa-project.org/wiki/AlsaInfo>`_.
1235 you, unless it's one of those 'issues of high priority' outlined earlier: in
1249 In case you performed a successful bisection, use the title of the change that
1251 also mention the commit id of the culprit. In case of an unsuccessful bisection,
1253 and the oldest where the issue occurs (say 5.8-rc1).
1256 (regressions@lists.linux.dev). In case the report needs to be filed to some web
1258 regressions list; CC the maintainer and the mailing list for the subsystem in
1262 When mailing or forwarding the report, in case of a successful bisection add the
1263 author of the culprit to the recipients; also CC everyone in the signed-off-by
1267 short-term risk to other users would arise if details were publicly disclosed.
1275 * If you were supposed to file the issue in a bug tracker make sure to mark
1280 In both cases make sure to also mail your report to the addresses the
1281 MAINTAINERS file lists in the section 'security contact'. Ideally directly CC
1282 them when sending the report by mail. If you filed it in a bug tracker, forward
1286 See 'Documentation/admin-guide/security-bugs.rst' for more information.
1290 --------------------------------
1293 outcome in one way or the other. Thus react publicly and in a timely manner
1301 to fix it, test it, and send it straight for integration in mainline while
1309 details, here are a few important things you need to keep in mind for this part
1316 **Always reply in public**: When you filed the issue in a bug tracker, always
1318 mailed reports always use the 'Reply-all' function when replying to any mails
1320 to your report: go to your mail applications 'Sent' folder and use 'reply-all'
1322 list(s) and everyone else that gets involved over time stays in the loop; it
1326 There are just two situations where a comment in a bug tracker or a 'Reply-all'
1332 information that needs to be kept private. In that case it's okay to send it
1333 in private to the developer that asked for it. But note in the ticket or a
1336 **Do research before asking for clarifications or help**: In this part of the
1340 Linux kernel sources to test if it helps. In some cases it will be fine sending
1343 consider asking in other places for advice. For example ask a friend or post
1348 are scattered around the globe and thus might be in a different time zone – one
1351 In general, kernel developers will take one to five business days to respond to
1361 Sometimes the maintainer might not be responding in a timely manner; other
1363 regression or not. In such cases raise your concerns on the mailing list and
1365 might be appropriate to get a higher authority involved. In case of a WiFi
1370 **Proactive testing**: Every time the first pre-release (the 'rc1') of a new
1371 mainline kernel version gets released, go and check if the issue is fixed there
1372 or if anything of importance changed. Mention the outcome in the ticket or in a
1373 mail you sent as reply to your report (make sure it has all those in the CC
1374 that up to that point participated in the discussion). This will show your
1387 Here are your duties in case you got replies to your report:
1391 issues are normally reported in public it could be anyone that's replying —
1392 including people that want to help, but in the end might guide you totally off
1396 the right people, as a reminder to the maintainer (see below) might be in order
1407 possible fix, try to test it in timely manner, too. But do it properly and make
1410 proposed patch with a fix was applied, but in fact wasn't. Things like that
1412 notice when the kernel with the fix behaves just as one without it.
1417 Some reports will not get any reaction from the responsible Linux kernel
1421 In these cases wait two (better: three) weeks before sending a friendly
1425 get the ball running somehow. If the report got out by mail, do that in the
1430 in the proper order.
1443 If the report was proper you can send a second reminder; in it ask for advice
1445 mail is shortly after the first pre-release (the 'rc1') of a new Linux kernel
1449 If the second reminder again results in no reaction within a week, try to
1450 contact a higher-level maintainer asking for advice: even busy maintainers by
1460 It's also possible that after some discussion in the bug tracker or on a list
1463 comes to Linux kernel development. This and several other reasons for not
1464 getting help are explained in 'Why some issues won't get any reaction or remain
1467 Don't get devastated if you don't find any help or if the issue in the end does
1468 not get solved: the Linux kernel is FLOSS and thus you can still help yourself.
1471 together that mentions how many you are and why this is something that in your
1474 easier. And with a bit of luck there might be someone in the team that knows a
1478 Reference for "Reporting regressions within a stable and longterm kernel line"
1479 ------------------------------------------------------------------------------
1482 a regression within a stable and longterm kernel line.
1487 *Check if the kernel developers still maintain the Linux kernel version
1488 line you care about: go to the front page of kernel.org and make sure it
1492 Most kernel version lines only get supported for about three months, as
1495 need to check if the kernel developers still support the version line you care
1498 Note, if kernel.org lists two stable version lines on the front page, you
1500 support for it is likely to be abandoned soon. Then it will get a "end-of-life"
1502 kernel.org front page for a week or two, but are unsuitable for testing and
1512 <https://lore.kernel.org/stable/>`_ for reports about an issue like yours. If
1520 kernel. Ensure this kernel is not tainted and still shows the problem, as
1522 problem with a vendor kernel, check a vanilla build of the last version
1525 Before investing any more time in this process you want to check if the issue
1526 was already fixed in the latest release of version line you're interested in.
1527 This kernel needs to be vanilla and shouldn't be tainted before the issue
1528 happens, as detailed outlined already above in the section "Install a fresh
1529 kernel for testing".
1531 Did you first notice the regression with a vendor kernel? Then changes the
1533 a recheck. Say something broke when you updated from 5.10.4-vendor.42 to
1534 5.10.5-vendor.43. Then after testing the latest 5.10 release as outlined in
1537 regression and you need switch back to the main step-by-step guide to report
1544 (stable@vger.kernel.org) and CC the Linux regressions mailing list
1545 (regressions@lists.linux.dev); if you suspect the cause in a particular
1551 When reporting a regression that happens within a stable or longterm kernel
1554 stable and regressions mailing list is all it takes; but in case you suspect
1555 the cause in a particular subsystem, CC its maintainers and its mailing list
1561 your distributor released a update from Linux kernel 5.10.5 to 5.10.8. Then as
1562 instructed above go and check the latest kernel from that version line, say
1566 first version where things broke. Mention it in the report and state that 5.10.9
1574 the document 'Documentation/admin-guide/bug-bisect.rst' for details how to
1575 perform one. In case of a successful bisection add the author of the culprit to
1576 the recipients; also CC everyone in the signed-off-by chain, which you find at
1580 Reference for "Reporting issues only occurring in older kernel version lines"
1581 -----------------------------------------------------------------------------
1584 reproduce your issue with a mainline kernel, but want to see it fixed in older
1591 might not get the issue solved in older releases: the fix might be too big
1594 Even small and seemingly obvious code-changes sometimes introduce new and
1597 within rules outlined in 'Documentation/process/stable-kernel-rules.rst'.
1603 version line your care about. In that case you'll have no other choice then to
1610 *Perform the first three steps in the section "Reporting issues only
1611 occurring in older kernel version lines" above.*
1613 You need to carry out a few steps already described in another section of this
1616 * Check if the kernel developers still maintain the Linux kernel version line
1627 *Search the Linux kernel version control system for the change that fixed
1628 the issue in mainline, as its commit message might tell you if the fix is
1631 or peer-review possible fixes; then check the discussions if the fix was
1633 all, join the newest discussion, asking if it's in the cards.*
1635 In a lot of cases the issue you deal with will have happened with mainline, but
1640 * First try to find the fix in the Git repository that holds the Linux kernel
1641 sources. You can do this with the web interfaces `on kernel.org
1642 <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/>`_
1645 log --grep=<pattern>``.
1650 Cc: <stable@vger.kernel.org> # 5.4+
1658 internet search engine as well as the archives for the `Linux kernel
1659 developers mailing list <https://lore.kernel.org/lkml/>`_. Also read the
1660 section `Locate kernel area that causes the issue` above and follow the
1661 instructions to find the subsystem in question: its bug tracker or mailing
1664 * If you see a proposed fix, search for it in the version control system as
1669 to live with the issue or switch to the kernel version line where the fix
1686 one option left: ask for advice. Do that in a mail you sent to the maintainers
1688 for the subsystem as well as the stable mailing list (stable@vger.kernel.org).
1697 will make sure of that. They and the other kernel developers will fix a lot of
1701 This is best explained with kernel developers that contribute to the Linux
1702 kernel in their spare time. Quite a few of the drivers in the kernel were
1718 something different in their life became way more important. In some cases
1720 to, as contributing to the Linux kernel is done on a voluntary basis. Abandoned
1721 drivers nevertheless remain in the kernel: they are still useful for people and
1725 work on the Linux kernel. Those contribute most changes these days. But their
1729 much time and energy in maintaining a Linux kernel driver for something they
1731 longer time period, but in new versions often leave support for old and rare
1739 spend on maintenance work on the upstream kernel. Sometimes maintainers also
1745 maintainers who are quite interested in fixing as many issues as possible.
1752 issues to the Linux kernel developers: the length and complexity of this
1761 fix it. You are free to do the same in a mostly informal way if you want
1763 linux-doc@vger.kernel.org and "sign-off" your contribution as
1764 Documentation/process/submitting-patches.rst outlines in the section "Sign
1765 your work - the Developer's Certificate of Origin".