Lines Matching +full:in +full:- +full:kernel
1 .. SPDX-License-Identifier: (GPL-2.0+ OR CC-BY-4.0)
13 <https://lore.kernel.org/lkml/>`_ and the `Linux stable mailing list
14 <https://lore.kernel.org/stable/>`_ archives for matching reports to join. If
16 <https://kernel.org/>`_. If it still shows the issue, report it to the stable
17 mailing list (stable@vger.kernel.org) and CC the regressions list
19 list for the subsystem in question.
21 In all other cases try your best guess which kernel part might be causing the
24 mailing list in CC. Check the destination's archives for matching reports;
25 search the `LKML <https://lore.kernel.org/lkml/>`_ and the web, too. If you
26 don't find any to join, install `the latest mainline kernel
27 <https://kernel.org/>`_. If the issue is present there, send a report.
29 The issue was fixed there, but you would like to see it resolved in a still
31 If it shows the problem, search for the change that fixed it in mainline and
32 check if backporting is in the works or was discarded; if it's neither, ask
35 **General remarks**: When installing and testing a kernel as outlined above,
36 ensure it's vanilla (IOW: not patched and not using add-on modules). Also make
37 sure it's built and running in a healthy environment and not already tainted
40 If you are facing multiple issues with the Linux kernel at once, report each
42 issue, like the kernel and the distro used. In case of a regression, CC the
44 to pin-point the culprit with a bisection; if you succeed, include its
45 commit-id and CC everyone in the sign-off-by chain.
51 Step-by-step guide how to report issues to the kernel maintainers
54 The above TL;DR outlines roughly how to report issues to the Linux kernel
58 step-by-step approach. It still tries to be brief for readability and leaves
59 out a lot of details; those are described below the step-by-step guide in a
60 reference section, which explains each of the steps in more detail.
62 Note: this section covers a few more aspects than the TL;DR and does things in
63 a slightly different order. That's in your interest, to make sure you notice
64 early if an issue that looks like a Linux kernel problem is actually caused by
65 something else. These steps thus help to ensure the time you invest in this
66 process won't feel wasted in the end:
68 * Are you facing an issue with a Linux kernel a hardware or software vendor
69 provided? Then in almost all cases you are better off to stop reading this
75 search engine; additionally, check the archives of the `Linux Kernel Mailing
76 List (LKML) <https://lore.kernel.org/lkml/>`_. If you find matching reports,
81 need special handling in some steps that are about to follow.
83 * Make sure it's not the kernel's surroundings that are causing the issue
89 kernel modules on-the-fly, which solutions like DKMS might be doing locally
92 * Check if your kernel was 'tainted' when the issue occurred, as the event
93 that made the kernel set this flag might be causing the issue you face.
98 needs to get reported to the kernel developers separately, unless they are
103 'Dealing with regressions within a stable and longterm kernel line'.
105 * Locate the driver or kernel subsystem that seems to be causing the issue.
107 time this won't be bugzilla.kernel.org, as issues typically need to be sent
110 * Search the archives of the bug tracker or mailing list in question
116 * Unless you are already running the latest 'mainline' Linux kernel, better
118 the latest 'stable' Linux can be an acceptable alternative in some
120 approach, but in that development phase it can be an even better idea to
125 * Ensure the kernel you just installed does not 'taint' itself when
128 * Reproduce the issue with the kernel you just installed. If it doesn't show
135 that hear about it for the first time. And if you learned something in this
139 decoding the kernel log to find the line of code that triggered the error.
145 issue. Always mention a few things: the latest kernel version you installed
147 reproduce the issue. Ideally, make the kernel's build configuration
157 special care which is explained in 'Special handling for high priority
161 outcome in one way or the other. Thus react publicly and in a timely manner
168 Reporting regressions within a stable and longterm kernel line
169 --------------------------------------------------------------
172 the point about regression within a stable or longterm kernel version line. You
178 * Check if the kernel developers still maintain the Linux kernel version
179 line you care about: go to the `front page of kernel.org
180 <https://kernel.org/>`_ and make sure it mentions
184 <https://lore.kernel.org/stable/>`_ for existing reports.
187 kernel. Ensure this kernel is not tainted and still shows the problem, as
189 problem with a vendor kernel, check a vanilla build of the last version
193 (stable@vger.kernel.org) and CC the Linux regressions mailing list
194 (regressions@lists.linux.dev); if you suspect the cause in a particular
200 The reference section below explains each of these steps in more detail.
203 Reporting issues only occurring in older kernel version lines
204 -------------------------------------------------------------
206 This subsection is for you, if you tried the latest mainline kernel as outlined
208 see the issue fixed in a still supported stable or longterm series or vendor
212 might not get the issue solved in older releases: the fix might be too big
215 * Perform the first three steps in the section "Dealing with regressions
216 within a stable and longterm kernel line" above.
218 * Search the Linux kernel version control system for the change that fixed
219 the issue in mainline, as its commit message might tell you if the fix is
222 or peer-review possible fixes; then check the discussions if the fix was
224 all, join the newest discussion, asking if it's in the cards.
231 The reference section below explains each of these steps in more detail.
234 Reference section: Reporting issues to the kernel maintainers
237 The detailed guides above outline all the major steps in brief fashion, which
247 * The Linux kernel developers are well aware this process is complicated and
249 that would require work in various places as well as some infrastructure,
254 request fixes from developers in the upstream Linux kernel community: such
255 contracts are completely outside the scope of the Linux kernel, its
257 anything such a contract guarantees in this context, not even if the
258 developer handling the issue works for the vendor in question. If you want
260 so, you might want to mention you'd like to see the issue fixed in the
261 upstream Linux kernel; motivate them by saying it's the only way to ensure
262 the fix in the end will get incorporated in all Linux distributions.
268 <http://www.catb.org/esr/faqs/smart-questions.html>`_, and `How to ask good
269 questions <https://jvns.ca/blog/good-questions/>`_.
272 issues to the Linux kernel developers.
275 Make sure you're using the upstream Linux kernel
276 ------------------------------------------------
278 *Are you facing an issue with a Linux kernel a hardware or software vendor
279 provided? Then in almost all cases you are better off to stop reading this
284 Like most programmers, Linux kernel developers don't like to spend time dealing
287 easily happen when it comes to the kernel and often leads to frustration on both
288 sides. That's because almost all Linux-based kernels pre-installed on devices
290 distributors are quite distant from the official Linux kernel as distributed by
291 kernel.org: these kernels from these vendors are often ancient from the point of
295 Linux kernel developers: an issue you face with one of them might have been
296 fixed by the Linux kernel developers months or years ago already; additionally,
300 report and, in case it turns out to be an upstream issue, fix it directly
301 upstream or forward the report there. In practice that often does not work out
303 vendor by installing the very latest Linux kernel core yourself. If that's an
304 option for you move ahead in this process, as a later step in this guide will
308 developers in fact are willing to handle reports about issues occurring with
309 vendor kernels. If they do in the end highly depends on the developers and the
310 issue in question. Your chances are quite good if the distributor applied only
311 small modifications to a kernel based on a recent Linux version; that for
314 with kernels from distributions shipping the latest stable kernel, as long as
316 regular Fedora releases, and openSUSE Tumbleweed. But keep in mind, you better
317 want to use a mainline Linux and avoid using a stable kernel for this
318 process, as outlined in the section 'Install a fresh kernel for testing' in more
322 or heavily modified vendor kernel to the upstream Linux developers. But note,
329 --------------------------------------
332 search engine; additionally, check the archives of the Linux Kernel Mailing
337 time for everyone involved, especially you as the reporter. So it's in your own
345 search the `Linux Kernel Mailing List (LKML) archives
346 <https://lore.kernel.org/lkml/>`_.
354 the name of the kernel driver or the name of the affected hardware component.
360 In case you find an existing report about your issue, join the discussion, as
362 important even when a fix is prepared or in its final stages already, as
367 Note, searching `bugzilla.kernel.org <https://bugzilla.kernel.org/>`_ might also
369 reports. If you find the latter, just keep in mind: most subsystems expect
370 reports in different places, as described below in the section "Check where you
373 the issue already got reported as outlined in this document and if not consider
378 -----------------------
382 need special handling in some steps that are about to follow.*
384 Linus Torvalds and the leading Linux kernel developers want to see some issues
386 handled slightly differently in the reporting process. Three type of cases
390 fine with one Linux kernel works worse or not at all with a newer version
392 Documentation/admin-guide/reporting-regressions.rst explains this in more
398 Documentation/process/security-bugs.rst before proceeding, as it
402 happens. That's for example the case when a Linux kernel corrupts the data it's
404 issue when the kernel suddenly stops working with an error message ('kernel
406 fatal error where the kernel stop itself) with a 'Oops' (a recoverable error),
407 as the kernel remains running after the latter.
411 ----------------------------
413 *Make sure it's not the kernel's surroundings that are causing the issue
416 Problems that look a lot like a kernel issue are sometimes caused by build or
420 * Use proven tools when building your kernel, as bugs in the compiler or the
421 binutils can cause the resulting kernel to misbehave.
426 potential kernel issue.
429 main memory for example can result in a multitude of issues that will
430 manifest itself in problems looking like kernel issues.
433 system in question with ``fsck``, as it might be damaged in a way that leads
434 to unexpected kernel behavior.
437 changed in parallel to updating the kernel. The problem for example might be
440 into a new kernel for the first time. Updating the systems BIOS or changing
441 something in the BIOS Setup can also lead to problems that on look a lot
442 like a kernel regression.
446 -----------------------
451 especially if you fiddle with crucial parts like the kernel of its operating
452 system. That's what you are about to do in this process. Thus, make sure to
458 Make sure your kernel doesn't get enhanced
459 ------------------------------------------
462 kernel modules on-the-fly, which solutions like DKMS might be doing locally
466 your kernel gets enhanced in any way. That's why you should remove or disable
467 mechanisms like akmods and DKMS: those build add-on kernel modules
468 automatically, for example when you install a new Linux kernel or boot it for
475 module not part of the Linux kernel. That why your might need to uninstall the
476 packages with such software to get rid of any 3rd party kernel module.
480 ------------------
482 *Check if your kernel was 'tainted' when the issue occurred, as the event
483 that made the kernel set this flag might be causing the issue you face.*
485 The kernel marks itself with a 'taint' flag when something happens that might
486 lead to follow-up errors that look totally unrelated. The issue you face might
487 be such an error if your kernel is tainted. That's why it's in your interest to
490 install the latest mainline kernel; you will need to check the taint flag again
491 then, as that's when it matters because it's the kernel the report will focus
494 On a running system is easy to check if the kernel tainted itself: if ``cat
495 /proc/sys/kernel/tainted`` returns '0' then the kernel is not tainted and
496 everything is fine. Checking that file is impossible in some situations; that's
497 why the kernel also mentions the taint status when it reports an internal
498 problem (a 'kernel bug'), a recoverable error (a 'kernel Oops') or a
499 non-recoverable error before halting operation (a 'kernel panic'). Look near
501 line starting with 'CPU:'. It should end with 'Not tainted' if the kernel was
505 If your kernel is tainted, study Documentation/admin-guide/tainted-kernels.rst
509 1. A recoverable error (a 'kernel Oops') occurred and the kernel tainted
510 itself, as the kernel knows it might misbehave in strange ways after that
511 point. In that case check your kernel or system log and look for a section
516 That's the first Oops since boot-up, as the '#1' between the brackets shows.
518 follow-up problem to that first Oops, even if both look totally unrelated.
523 the cause for the Oops might already be fixed in the newer Linux kernel
524 version you are going to install later in this process.
526 2. Your system uses a software that installs its own kernel modules, for
527 example Nvidia's proprietary graphics driver or VirtualBox. The kernel
529 they are Open Source): they sometimes cause errors in unrelated kernel
532 Linux kernel developers. Most of the time the easiest way to do that is:
536 3. The kernel also taints itself when it's loading a module that resides in
537 the staging tree of the Linux kernel source. That's a special area for
538 code (mostly drivers) that does not yet fulfill the normal Linux kernel
540 obviously okay if the kernel is tainted; just make sure the module in
541 question is the only reason for the taint. If the issue happens in an
543 by specifying ``foo.blacklist=1`` as kernel parameter (replace 'foo' with
544 the name of the module in question).
548 -------------------------------
553 needs to get reported to the kernel developers separately, unless they are
558 various issues in one report also makes it quite difficult for others to tear
559 it apart. Hence, only combine issues in one report if they are very strongly
563 happens with other kernel versions. Therefore, it will make your work easier if
569 to ignore this advice if you are experienced enough to tell a one-time error
570 due to faulty hardware apart from a kernel issue that rarely happens and thus
574 Regression in stable or longterm kernel?
575 ----------------------------------------
579 'Dealing with regressions within a stable and longterm kernel line'.*
581 Regression within a stable and longterm kernel version line are something the
583 regression in the main development branch, as they can quickly affect a lot of
586 regressions with newer kernel version line (say something broke when switching
591 -----------------------------------------
593 *Locate the driver or kernel subsystem that seems to be causing the issue.
595 time this won't be bugzilla.kernel.org, as issues typically need to be sent
598 It's crucial to send your report to the right people, as the Linux kernel is a
605 Problem is: the Linux kernel lacks a central bug tracker where you can simply
609 kernel developers and experts. For everybody else the MAINTAINERS file is the
615 the WiFi in your Laptop suddenly misbehaves after updating the kernel. In that
616 case it's likely an issue in the WiFi driver. Obviously it could also be some
624 In case of a problem with the WiFi driver you for example might want to look at
625 the output of ``lspci -k``, as it lists devices on the PCI/PCIe bus and the
626 kernel module driving it::
628 [user@something ~]$ lspci -k
632 Kernel driver in use: ath10k_pci
633 Kernel modules: ath10k_pci
637 other internal bus. In those cases you might want to check your WiFi manager or
642 …[user@something ~]$ realpath --relative-to=/sys/module/ /sys/class/net/wlp58s0/device/driver/module
645 In case tricks like these don't bring you any further, try to search the
646 internet on how to narrow down the driver or subsystem in question. And if you
650 Once you know the driver or subsystem, you want to search for it in the
651 MAINTAINERS file. In the case of 'ath10k_pci' you won't find anything, as the
660 Web-page: https://wireless.wiki.kernel.org/en/users/Drivers/ath10k
661 SCM: git git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git
665 MAINTAINERS file found in the root of the Linux source tree. 'Mail:' for
680 Linux kernel development is completely driven by mail. Very few subsystems use
681 a bug tracker, and only some of those rely on bugzilla.kernel.org.
683 In this and many other cases you thus have to look for lines starting with
688 issue reports sent by email, make sure to add the Linux Kernel Mailing List
689 (LKML) <linux-kernel@vger.kernel.org> to CC. Don't omit either of the mailing
701 called with a path to the source code in question. For drivers compiled as
704 …$ modinfo ath10k_pci | grep filename | sed 's!/lib/modules/.*/kernel/!!; s!filename:!!; s!\.ko\(\|…
709 $ ./scripts/get_maintainer.pl -f drivers/net/wireless/ath/ath10k*
713 linux-wireless@vger.kernel.org (open list:NETWORKING DRIVERS (WIRELESS))
714 netdev@vger.kernel.org (open list:NETWORKING DRIVERS)
715 linux-kernel@vger.kernel.org (open list)
719 the code as well as the Linux Kernel Mailing List (LKML). In this case you thus
721 'ath10k@lists.infradead.org' and 'linux-kernel@vger.kernel.org' in CC.
723 Note: in case you cloned the Linux sources with git you might want to call
724 ``get_maintainer.pl`` a second time with ``--git``. The script then will look
725 at the commit history to find which people recently worked on the code in
727 can easily send you in a wrong direction. That for example happens quickly in
729 modified during tree-wide cleanups by developers that do not care about the
734 ---------------------------------------
736 *Search the archives of the bug tracker or mailing list in question
744 often find its archives on `lore.kernel.org <https://lore.kernel.org/>`_.
746 But some list are hosted in different places. That for example is the case for
747 the ath10k WiFi driver used as example in the previous step. But you'll often
753 quite a few other lists miss a way to search the archives. In those cases use a
758 It's also wise to check the internet, LKML and maybe bugzilla.kernel.org again
759 at this point. If your report needs to be filed in a bug tracker, you may want
770 Install a fresh kernel for testing
771 ----------------------------------
773 *Unless you are already running the latest 'mainline' Linux kernel, better
775 the latest 'stable' Linux can be an acceptable alternative in some
777 approach, but in that development phase it can be an even better idea to
782 As mentioned in the detailed explanation for the first step already: Like most
783 programmers, Linux kernel developers don't like to spend time dealing with
785 waste everybody's time, especially yours. That's why it's in everybody's
791 In the scope of the kernel "latest upstream" normally means:
793 * Install a mainline kernel; the latest stable kernel can be an option, but
796 explains all of this in more detail.
798 * The over next subsection describes way to obtain and install such a kernel.
799 It also outlines that using a pre-compiled kernel are fine, but better are
801 kernel.org <https://kernel.org/>`_ and not modified or enhanced in any way.
806 Head over to `kernel.org <https://kernel.org/>`_ to find out which version you
809 mainline, which most of the time will point to a pre-release with a version
810 number like '5.8-rc2'. If that's the case, you'll want to use this mainline
811 kernel for testing, as that where all fixes have to be applied first. Do not let
815 In about two out of every nine to ten weeks, mainline might point you to a
817 suspending the reporting process until the first pre-release of the next
818 version (5.8-rc1) shows up on kernel.org. That's because the Linux development
819 cycle then is in its two-week long 'merge window'. The bulk of the changes and
821 more risky to use mainline during this period. Kernel developers are also often
825 a newer kernel version anyway, as outlined below in the section 'Duties after
829 to that if you're dealing with something that shouldn't wait. In that case
830 consider obtaining the latest mainline kernel via git (see below) or use the
831 latest stable version offered on kernel.org. Using that is also acceptable in
832 case mainline for some reason does currently not work for you. An in general:
836 Better avoid using the latest stable kernel outside merge windows, as all fixes
838 kernel is so important: any issue you want to see fixed in older version lines
839 needs to be fixed in mainline first before it can get backported, which can
848 it fixed in older version lines, if that's in the cards for the fix in question.
850 How to obtain a fresh Linux kernel
853 **Using a pre-compiled kernel**: This is often the quickest, easiest, and safest
854 way for testing — especially is you are unfamiliar with the Linux kernel. The
855 problem: most of those shipped by distributors or add-on repositories are build
860 But you are in luck if you are using a popular Linux distribution: for quite a
862 latest mainline or stable Linux built as vanilla kernel. It's totally okay to
865 versions as offered on kernel.org. The packages are likely unsuitable if they
869 Please note that you might need to build your own kernel manually later: that's
870 sometimes needed for debugging or testing fixes, as described later in this
871 document. Also be aware that pre-compiled kernels might lack debug symbols that
872 are needed to decode messages the kernel prints when a panic, Oops, warning, or
874 kernel yourself (see the end of this subsection and the section titled 'Decode
878 often best served by obtaining the latest Linux kernel sources straight from the
879 `official development repository on kernel.org
880 <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/>`_.
881 Those are likely a bit ahead of the latest mainline pre-release. Don't worry
882 about it: they are as reliable as a proper pre-release, unless the kernel's
883 development cycle is currently in the middle of a merge window. But even then
887 downloading the sources as tarball from `kernel.org <https://kernel.org/>`_.
889 How to actually build a kernel is not described here, as many websites explain
891 those how-to's that suggest to use ``make localmodconfig``, as that tries to
892 pick up the configuration of your current kernel and then tries to adjust it
893 somewhat for your system. That does not make the resulting kernel any better,
896 Note: If you are dealing with a panic, Oops, warning, or BUG from the kernel,
897 please try to enable CONFIG_KALLSYMS when configuring your kernel.
901 build a kernel by quite a bit. But that's worth it, as these options will allow
903 section 'Decode failure messages' below explains this in more detail.
905 But keep in mind: Always keep a record of the issue encountered in case it is
911 ------------------
913 *Ensure the kernel you just installed does not 'taint' itself when
916 As outlined above in more detail already: the kernel sets a 'taint' flag when
917 something happens that can lead to follow-up errors that look totally
918 unrelated. That's why you need to check if the kernel you just installed does
919 not set this flag. And if it does, you in almost all the cases needs to
924 Reproduce issue with the fresh kernel
925 -------------------------------------
927 *Reproduce the issue with the kernel you just installed. If it doesn't show
931 Check if the issue occurs with the fresh Linux kernel version you just
933 line and abandoning your plan to report the issue. But keep in mind that other
934 users might still be plagued by it, as long as it's not fixed in either stable
935 and longterm version from kernel.org (and thus vendor kernels derived from
937 head over to the section "Details about reporting issues only occurring in
938 older kernel version lines" below.
942 ---------------------------------------
947 that hear about it for the first time. And if you learned something in this
952 thus easy to understand in written form. Include all important details, but at
955 In this in the previous steps you likely have learned a thing or two about the
961 -----------------------
964 decoding the kernel log to find the line of code that triggered the error.*
966 When the kernel detects an internal problem, it will log some information about
967 the executed code. This makes it possible to pinpoint the exact line in the
970 your kernel. If you did so, consider to decode the information from the
971 kernel's log. That will make it a lot easier to understand what lead to the
975 Decoding can be done with a script you find in the Linux source tree. If you
976 are running a kernel you compiled yourself earlier, call it like this::
978 …[user@something ~]$ sudo dmesg | ./linux-5.10.5/scripts/decode_stacktrace.sh ./linux-5.10.5/vmlinux
980 If you are running a packaged vanilla kernel, you will likely have to install
985 [user@something ~]$ sudo dmesg | ./linux-5.10.5/scripts/decode_stacktrace.sh \
986 /usr/lib/debug/lib/modules/5.10.10-4.1.x86_64/vmlinux /usr/src/kernels/5.10.10-4.1.x86_64/
989 the code the kernel was executing when the error occurred::
995 …[ 68.387301] RIP: 0010:test_module_init (/home/username/linux-5.10.5/test-module/test-module.c:1…
997 In this case the executed code was built from the file
998 '~/linux-5.10.5/test-module/test-module.c' and the error occurred by the
999 instructions found in line '16'.
1001 The script will similarly decode the addresses mentioned in the section
1004 the code section the kernel was executing.
1007 reason for it in the report. If you're lucky, it might not be needed. And if it
1009 of several ways to decode kernel stack traces. Sometimes different steps will
1011 needed in your case, developers will tell you what to do.
1015 ----------------------------
1020 Linux lead developer Linus Torvalds insists that the Linux kernel never
1031 Documentation/admin-guide/bug-bisect.rst describes in detail. That process
1032 will often require you to build about ten to twenty kernel images, trying to
1035 Thanks to a 'binary search' this will lead you to the one commit in the source
1041 Note, a bisection needs a bit of know-how, which not everyone has, and quite a
1044 don't want to go down that route at least find out which mainline kernel
1046 5.5.15 to 5.8.4, then try at least all the mainline releases in that area (5.6,
1048 regression in a stable or longterm kernel, avoid testing versions which number
1051 version which introduced the regression, feel free to move on in the reporting
1052 process. But keep in mind: it depends on the issue at hand if the developers
1058 the kernel and not by something else, as outlined above already.
1060 In the whole process keep in mind: an issue only qualifies as regression if the
1061 older and the newer kernel got built with a similar configuration. This can be
1062 achieved by using ``make olddefconfig``, as explained in more detail by
1063 Documentation/admin-guide/reporting-regressions.rst; that document also
1069 -------------------------
1072 issue. Always mention a few things: the latest kernel version you installed
1074 reproduce the issue. Ideally, make the kernel's build configuration
1084 special care which is explained in 'Special handling for high priority
1088 that is partly explained by the three documents linked to in the preface above.
1090 things specific to the Linux kernel.
1098 and write the detailed report first. ;-)
1103 Describe in detail how your issue happens with the fresh vanilla kernel you
1104 installed. Try to include the step-by-step instructions you wrote and optimized
1105 earlier that outline how you and ideally others can reproduce the issue; in
1113 * the output from ``cat /proc/version``, which contains the Linux kernel
1119 * the architecture of the CPU and the operating system (``uname -mi``)
1122 subject and the commit-id of the change that is causing it.
1124 In a lot of cases it's also wise to make two more things available to those
1127 * the configuration used for building your Linux kernel (the '.config' file)
1129 * the kernel's messages that you get from ``dmesg`` written to a file. Make
1130 sure that it starts with a line like 'Linux version 5.8-1
1133 boot phase already got discarded. In this case instead consider using
1134 ``journalctl -b 0 -k``; alternatively you can also reboot, reproduce the
1138 your report. If you are filing the issue in a bug tracker then attach them to
1143 service, a ticket created just for this purpose on `bugzilla.kernel.org
1144 <https://bugzilla.kernel.org/>`_, ...) and include a link to them in your
1150 * Put the files aside and mention you will send them later in individual
1152 went out. ;-)
1160 * If you are dealing with a 'warning', an 'OOPS' or a 'panic' from the kernel,
1170 nothing in common. Hence, in such cases add the exact model number, which
1176 * Mention the relevant software in use. If you have problems with loading
1177 modules, you want to mention the versions of kmod, systemd, and udev in use.
1179 libdrm and Mesa; also specify your Wayland compositor or the X-Server and
1181 corresponding filesystem utilities (e2fsprogs, btrfs-progs, xfsprogs, ...).
1183 * Gather additional information from the kernel that might be of interest. The
1184 output from ``lspci -nn`` will for example help others to identify what
1186 make the output from ``sudo lspci -vvv`` available, as that provides
1191 information. One such tool is ``alsa-info.sh`` `which the audio/sound
1192 subsystem developers provide <https://www.alsa-project.org/wiki/AlsaInfo>`_.
1221 you, unless it's one of those 'issues of high priority' outlined earlier: in
1235 In case you performed a successful bisection, use the title of the change that
1237 also mention the commit id of the culprit. In case of an unsuccessful bisection,
1239 and the oldest where the issue occurs (say 5.8-rc1).
1242 (regressions@lists.linux.dev). In case the report needs to be filed to some web
1244 regressions list; CC the maintainer and the mailing list for the subsystem in
1248 When mailing or forwarding the report, in case of a successful bisection add the
1249 author of the culprit to the recipients; also CC everyone in the signed-off-by
1253 short-term risk to other users would arise if details were publicly disclosed.
1261 * If you were supposed to file the issue in a bug tracker make sure to mark
1266 In both cases make sure to also mail your report to the addresses the
1267 MAINTAINERS file lists in the section 'security contact'. Ideally directly CC
1268 them when sending the report by mail. If you filed it in a bug tracker, forward
1272 See Documentation/process/security-bugs.rst for more information.
1276 --------------------------------
1279 outcome in one way or the other. Thus react publicly and in a timely manner
1287 to fix it, test it, and send it straight for integration in mainline while
1295 details, here are a few important things you need to keep in mind for this part
1302 **Always reply in public**: When you filed the issue in a bug tracker, always
1304 mailed reports always use the 'Reply-all' function when replying to any mails
1306 to your report: go to your mail applications 'Sent' folder and use 'reply-all'
1308 list(s) and everyone else that gets involved over time stays in the loop; it
1312 There are just two situations where a comment in a bug tracker or a 'Reply-all'
1318 information that needs to be kept private. In that case it's okay to send it
1319 in private to the developer that asked for it. But note in the ticket or a
1322 **Do research before asking for clarifications or help**: In this part of the
1326 Linux kernel sources to test if it helps. In some cases it will be fine sending
1329 consider asking in other places for advice. For example ask a friend or post
1334 are scattered around the globe and thus might be in a different time zone – one
1337 In general, kernel developers will take one to five business days to respond to
1347 Sometimes the maintainer might not be responding in a timely manner; other
1349 regression or not. In such cases raise your concerns on the mailing list and
1351 might be appropriate to get a higher authority involved. In case of a WiFi
1356 **Proactive testing**: Every time the first pre-release (the 'rc1') of a new
1357 mainline kernel version gets released, go and check if the issue is fixed there
1358 or if anything of importance changed. Mention the outcome in the ticket or in a
1359 mail you sent as reply to your report (make sure it has all those in the CC
1360 that up to that point participated in the discussion). This will show your
1373 Here are your duties in case you got replies to your report:
1377 issues are normally reported in public it could be anyone that's replying —
1378 including people that want to help, but in the end might guide you totally off
1382 the right people, as a reminder to the maintainer (see below) might be in order
1393 possible fix, try to test it in timely manner, too. But do it properly and make
1396 proposed patch with a fix was applied, but in fact wasn't. Things like that
1398 notice when the kernel with the fix behaves just as one without it.
1403 Some reports will not get any reaction from the responsible Linux kernel
1407 In these cases wait two (better: three) weeks before sending a friendly
1411 get the ball running somehow. If the report got out by mail, do that in the
1416 in the proper order.
1429 If the report was proper you can send a second reminder; in it ask for advice
1431 mail is shortly after the first pre-release (the 'rc1') of a new Linux kernel
1435 If the second reminder again results in no reaction within a week, try to
1436 contact a higher-level maintainer asking for advice: even busy maintainers by
1446 It's also possible that after some discussion in the bug tracker or on a list
1449 comes to Linux kernel development. This and several other reasons for not
1450 getting help are explained in 'Why some issues won't get any reaction or remain
1453 Don't get devastated if you don't find any help or if the issue in the end does
1454 not get solved: the Linux kernel is FLOSS and thus you can still help yourself.
1457 together that mentions how many you are and why this is something that in your
1460 easier. And with a bit of luck there might be someone in the team that knows a
1464 Reference for "Reporting regressions within a stable and longterm kernel line"
1465 ------------------------------------------------------------------------------
1468 a regression within a stable and longterm kernel line.
1473 *Check if the kernel developers still maintain the Linux kernel version
1474 line you care about: go to the front page of kernel.org and make sure it
1478 Most kernel version lines only get supported for about three months, as
1481 need to check if the kernel developers still support the version line you care
1484 Note, if kernel.org lists two stable version lines on the front page, you
1486 support for it is likely to be abandoned soon. Then it will get a "end-of-life"
1488 kernel.org front page for a week or two, but are unsuitable for testing and
1498 <https://lore.kernel.org/stable/>`_ for reports about an issue like yours. If
1506 kernel. Ensure this kernel is not tainted and still shows the problem, as
1508 problem with a vendor kernel, check a vanilla build of the last version
1511 Before investing any more time in this process you want to check if the issue
1512 was already fixed in the latest release of version line you're interested in.
1513 This kernel needs to be vanilla and shouldn't be tainted before the issue
1514 happens, as detailed outlined already above in the section "Install a fresh
1515 kernel for testing".
1517 Did you first notice the regression with a vendor kernel? Then changes the
1519 a recheck. Say something broke when you updated from 5.10.4-vendor.42 to
1520 5.10.5-vendor.43. Then after testing the latest 5.10 release as outlined in
1523 regression and you need switch back to the main step-by-step guide to report
1530 (stable@vger.kernel.org) and CC the Linux regressions mailing list
1531 (regressions@lists.linux.dev); if you suspect the cause in a particular
1537 When reporting a regression that happens within a stable or longterm kernel
1540 stable and regressions mailing list is all it takes; but in case you suspect
1541 the cause in a particular subsystem, CC its maintainers and its mailing list
1547 your distributor released a update from Linux kernel 5.10.5 to 5.10.8. Then as
1548 instructed above go and check the latest kernel from that version line, say
1552 first version where things broke. Mention it in the report and state that 5.10.9
1560 the document Documentation/admin-guide/bug-bisect.rst for details how to
1561 perform one. In case of a successful bisection add the author of the culprit to
1562 the recipients; also CC everyone in the signed-off-by chain, which you find at
1566 Reference for "Reporting issues only occurring in older kernel version lines"
1567 -----------------------------------------------------------------------------
1570 reproduce your issue with a mainline kernel, but want to see it fixed in older
1577 might not get the issue solved in older releases: the fix might be too big
1580 Even small and seemingly obvious code-changes sometimes introduce new and
1583 within rules outlined in Documentation/process/stable-kernel-rules.rst.
1589 version line your care about. In that case you'll have no other choice then to
1596 *Perform the first three steps in the section "Reporting issues only
1597 occurring in older kernel version lines" above.*
1599 You need to carry out a few steps already described in another section of this
1602 * Check if the kernel developers still maintain the Linux kernel version line
1613 *Search the Linux kernel version control system for the change that fixed
1614 the issue in mainline, as its commit message might tell you if the fix is
1617 or peer-review possible fixes; then check the discussions if the fix was
1619 all, join the newest discussion, asking if it's in the cards.*
1621 In a lot of cases the issue you deal with will have happened with mainline, but
1626 * First try to find the fix in the Git repository that holds the Linux kernel
1627 sources. You can do this with the web interfaces `on kernel.org
1628 <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/>`_
1631 log --grep=<pattern>``.
1636 Cc: <stable@vger.kernel.org> # 5.4+
1644 internet search engine as well as the archives for the `Linux kernel
1645 developers mailing list <https://lore.kernel.org/lkml/>`_. Also read the
1646 section `Locate kernel area that causes the issue` above and follow the
1647 instructions to find the subsystem in question: its bug tracker or mailing
1650 * If you see a proposed fix, search for it in the version control system as
1655 to live with the issue or switch to the kernel version line where the fix
1672 one option left: ask for advice. Do that in a mail you sent to the maintainers
1674 for the subsystem as well as the stable mailing list (stable@vger.kernel.org).
1683 will make sure of that. They and the other kernel developers will fix a lot of
1687 This is best explained with kernel developers that contribute to the Linux
1688 kernel in their spare time. Quite a few of the drivers in the kernel were
1704 something different in their life became way more important. In some cases
1706 to, as contributing to the Linux kernel is done on a voluntary basis. Abandoned
1707 drivers nevertheless remain in the kernel: they are still useful for people and
1711 work on the Linux kernel. Those contribute most changes these days. But their
1715 much time and energy in maintaining a Linux kernel driver for something they
1717 longer time period, but in new versions often leave support for old and rare
1725 spend on maintenance work on the upstream kernel. Sometimes maintainers also
1731 maintainers who are quite interested in fixing as many issues as possible.
1738 issues to the Linux kernel developers: the length and complexity of this
1745 end-of-content
1749 he'll fix it. You are free to do the same in a mostly informal way if you
1751 linux-doc@vger.kernel.org and "sign-off" your contribution as
1752 Documentation/process/submitting-patches.rst outlines in the section "Sign
1753 your work - the Developer's Certificate of Origin".
1755 This text is available under GPL-2.0+ or CC-BY-4.0, as stated at the top
1756 of the file. If you want to distribute this text under CC-BY-4.0 only,
1757 please use "The Linux kernel developers" for author attribution and link
1759 …https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/Documentation/admin-guide…
1761 Note: Only the content of this RST file as found in the Linux kernel sources
1762 is available under CC-BY-4.0, as versions of this text that were processed
1763 (for example by the kernel's build system) might contain content taken from