Hello, Planet Debian

Hey, there. This is long overdue: my entry in Planet Debian! I’m creating this post because, until now, I didn’t have a debian tag in my blog! Well, not anymore.

Stay tunned!


Dreaming...

Back then, I still wanted to write something. Back then, life was different, and I had another view of myself and of others. Back then, my house of cards was still standing, giving the impression that it was safe and sound, that its foundation was solid, and that nothing would shake it. But that was back then.

Right now, I have lost my will and my power to concentrate, to focus on what really matters, because what really matters is still undefined. Right now, things don’t seem to fit as they once did; the vision blurs and I am not so sure what it is that I should be doing but am not. Right now, my self has become another one. Someone that doesn’t remind me of anybody in particular.

Struggling, defining, living and knowing. These are constant words, constant feelings and actions that live with me. Who am I? What do I like? What do I don’t like? Am I good in what I thought I was good? Am I feeling what I think I’m feeling?

This is more than the impostor syndrome. But it is less than the Stockholm syndrome. It’s somewhere in between, or maybe nowhere. When I woke up and decided to keep going, I knew it was a temporary decision. It still is. I still have to find what I missed, or what I have never found. What to do? Too hard of a question to answer right now. Here’s hoping that time will help me with this hard, but long-wanted task.


Combater

Às vezes, é preciso combater. É preciso dizer que o outro está errado, que ele está falando besteira sobre um assunto que não conhece (e não quer conhecer). É preciso dizer o que é ético, o que é certo. É preciso discernir tudo o que é errado e anti-ético, imoral, e que faz mal. É preciso combater o ódio, muitas vezes com amor, outras tantas com força e integridade.

É preciso falar praquele ignorante que ele não sabe o que é Software Livre. É preciso dizer que o Software Livre é muito maior do que o GNU, muito maior do que uma pessoa ou do que suas declarações. É preciso dizer que o ignorante tornou-se troll. É preciso dizer que ele não sabe o que fala, e que deve calar-se. É preciso deixar que ele viva sua adolescência conturbada e por vezes medíocre, mas tomando cuidado para que isso não influencie outras pessoas ignorantes a tornarem-se trolls também. É preciso que esse troll saia do Twitter, saia do BR-[GNU/]Linux, saia dos fóruns movidos a coisas proprietárias; ou talvez seja preciso que ele fique lá, destilando seu ódio, veneno e ignorância para seus semelhantes.

É preciso combater o liberalismo de fachada, que é um veículo para o ódio. É preciso combater o ódio. É preciso combater a ignorância, novamente. É preciso combater o reacionarismo disfarçado de “livre mercado”, é preciso combater a falta de bom senso que ocorre quando se generaliza um partido político por um comportamento, é preciso combater o comportamento, é preciso fazer progresso social sempre, é preciso parar de se importar tanto com aqueles que não se importam.

É preciso combater o pastor ignorante. É preciso combater a ignorância, uma terceira vez. É preciso combater a “trollagem” do pastor, dos fiéis e dos simpatizantes a eles. É preciso combater a onda de “radicalismo conservador” que aflige a todos. É preciso combater a falta de amor ao próximo e o excesso de arrogância. É preciso combater as falsas palavras divinas, as falsas vontades de uma entidade, as falsas aglomerações públicas em torno de um erro.

É preciso combater o apresentador idiota, ignorante e presunçoso. É preciso combater o que se destila de ódio naquele país, porque nem todos têm um soro contra veneno de cobra criada. É preciso combater a ignorância, novamente, porque ela é o caminho mais fácil para o ódio, e o ódio retroalimenta a ignorância num ciclo difícil de ser quebrado. É preciso ensinar a aprender, e aprender a ensinar. É preciso combater a preguiça, essa desculpa tão usada e repetida que chega a dar preguiça de combatê-la. É preciso sair do sofá, mas não para ir para o Twitter ou Facebook; é preciso sair do sofá e ser crítico o suficiente para saber o que se deve fazer, porque não sou eu quem vou falar.


Memory mappings, core dumps, GDB and Linux

After spending the last weeks struggling with this, I decided to write a blog post. First, what is “this” that you are talking about? The answer is: Linux kernel’s concept of memory mapping. I found it utterly confused, beyond my expectations, and so I believe that a blog post is the write way to (a) preserve and (b) share this knowledge. So, let’s do it!

First things first

First, I cannot begin this post without a few acknowledgements and “thank you’s”. The first goes to Oleg Nesterov (sorry, I could not find his website), a Linux kernel guru who really helped me a lot through the whole task. Another “thank you” goes to Jan Kratochvil, who also provided valuable feedback by commenting my GDB patch. Now, back to the point.

The task

The task was requested here: GDB needed to respect the /proc/<PID>/coredump_filter file when generating a coredump (i.e., when you use the gcore command).

Currently, GDB has his own coredump mechanism implemented which, despite its limitations and bugs, has been around for quite some time. However, and maybe you don’t know that, but the Linux kernel has its own algorithm for generating the corefile of a process. And unfortunately, GDB and Linux were not really following the same standards here…

So, in the end, the task was about synchronizing GDB and Linux. To do that, I first had to decipher the contents of the /proc/<PID>/smaps file.

The /proc/<PID>/smaps file

This special file, generated by the Linux kernel when you read it, contains detailed information about each memory mapping of a certain process. Some of the fields on this file are documented in the proc(5) manpage, but others are missing there (asking for a patch!). Here is an explanation of everything I needed:

  • The first line of each memory mapping has the following format:

    The fields here are:

    a) address is the address range, in the process’ address space, that the mapping occupies. This part was already treated by GDB, so I did not have to worry about it.

    b) perms is a set of permissions (r ead, w rite, e x ecute, s hared, p rivate [COW – copy-on-write]) applied to the memory mapping. GDB was already dealing with rwx permissions, but I needed to include the p flag as well. I also made GDB ignore the mappings that did not have the r flag active, because it does not make sense to dump something that you cannot read.

    c) offset is the offset into the applied to the file, if the mapping is file-backed (see below). GDB already handled this correctly.

    d) dev is the device (major:minor) related to the file, if there is one. GDB already handled this correctly, though I was using this field for more things (continue reading).

    e) inode is the inode on the device above. The value of zero means that no inode is associated with the memory mapping. Nothing to do here.

    f) pathname is the file associate with this mapping, if there is one. This is one of the most important fields that I had to use, and one of the most complicated to understand completely. GDB now uses this to heuristically identify whether the mapping is anonymous or not.

  • GDB is now also interested in Anonymous: and AnonHugePages: fields from the smaps file. Those fields represent the content of anonymous data on the mapping; if GDB finds that this content is greater than zero, this means that the mapping is anonymous.

  • The last, but perhaps most important field, is the VmFlags: field. It contains a series of two-letter flags that provide very useful information about the mapping. A description of the fields is: a) sh: the mapping is shared (VM_SHARED) b) dd: this mapping should not be dumped in a corefile (VM_DONTDUMP) c) ht: this is HugeTLB mapping

With that in hands, the following task was to be able to determine whether a memory mapping is anonymous or file-backed, private or shared.

Types of memory mappings

There can be four types of memory mappings:

  1. Anonymous private mapping
  2. Anonymous shared mapping
  3. File-backed private mapping
  4. File-backed shared mapping

It should be possible to uniquely identify each mapping based on the information provided by the smaps file; however, you will see that this is not always the case. Below, I will explain how to determine each of the four characteristics that define a mapping.

Anonymous

A mapping is anonymous if one of these conditions apply:

  1. The pathname associated with it is either /dev/zero (deleted), /SYSV%08x (deleted), or <filename> (deleted) (see below).
  2. There is content in the Anonymous: or in the AnonHugePages: fields of the mapping in the smaps file.

A special explanation is needed for the <filename> (deleted) case. It is not always guaranteed that it identifies an anonymous mapping; in fact, it is possible to have the (deleted) part for file-backed mappings as well (say, when you are running a program that uses shared libraries, and those shared libraries have been removed because of an update, for example). However, we are trying to mimic the behavior of the Linux kernel here, which checks to see if a file has no hard links associated with it (and therefore is truly deleted).

Although it may be possible for the userspace to do an extensive check (by stat ing the file, for example), the Linux kernel certainly could give more information about this.

File-backed

A mapping is file-backed (i.e., not anonymous) if:

  1. The pathname associated with it contains a <filename>, without the (deleted) part.

As has been explained above, a mapping whose pathname contains the (deleted) string could still be file-backed, but we decide to consider it anonymous.

It is also worth mentioning that a mapping can be simultaneously anonymous and file-backed: this happens when the mapping contains a valid pathname (without the (deleted) part), but also contains Anonymous: or AnonHugePages: contents.

Private

A mapping is considered to be private (i.e., not shared) if:

  1. In the absence of the VmFlags field (in the smaps file), its permission field has the flag p.
  2. If the VmFlags field is present, then the mapping is private if we do not find the sh flag there.

Shared

A mapping is shared (i.e., not private) if:

  1. In the absence of VmFlags in the smaps file, the permission field of the mapping does not have the p flag. Not having this flag actually means VM_MAYSHARE and not necessarily VM_SHARED (which is what we want), but it is the best approximation we have.
  2. If the VmFlags field is present, then the mapping is shared if we find the sh flag there.

The patch

With all that in mind, I hacked GDB to improve the coredump mechanism for GNU/Linux operating systems. The main function which decides the memory mappings that will or will not be dumped on GNU/Linux is linux_find_memory_regions_full; the Linux kernel obviously uses its own function, vma_dump_size, to do the same thing.

Linux has one advantage: it is a kernel, and therefore has much more knowledge about processes’ internals than a userspace program. For example, inside Linux it is trivial to check if a file marked as “(deleted)” in the output of the smaps file has no hard links associated with it (and therefore is not really deleted); the same operation on userspace, however, would require root access to inspect the contents of the /proc/<PID>/map_files/ directory.

The case described above, if you remember, is something that impacts the ability to tell whether a mapping is anonymous or not. I am talking to the Linux kernel guys to see if it is possible to export this information directly via the smaps file, instead of having to do the current heuristic.

While doing this work, some strange behaviors were found in the Linux kernel. Oleg is working on them, along with other Linux hackers. From our side, there is still room for improvement on this code. The first thing I can think of is to improve the heuristics for finding anonymous mappings. Another relatively easy thing to do would be to let the user specify a value for coredump_filter on the command line, without editing the /proc file. And of course, keep this code always updated with its counterpart in the Linux kernel.

Upstream discussions and commit

If you are interested, you can see the discussions that happened upstream by going to this link. This is the fourth (and final) submission of the patch; you should be able to find the other submissions in the archive.

The final commit can be found in the official repository.


Fazendo a Diferença

Deu saudade de escrever em português :-). E deu saudade, também, de fazer algum post mais “filosófico”.

Não sei dizer o porquê, mas às vezes tenho uma mania besta: gosto de ficar procurando “sarna pra me coçar”. Em outras palavras, eu fico procurando coisas que me deixam mal, mesmo sabendo que vou ficar mal depois de vê-las.

Não tenho explicação pra esse comportamento. É algo meio sabotador, meio sofredor, meio… Não sei. Às vezes, quando me vejo novamente nesse ciclo vicioso, consigo parar. No entanto, na maioria das vezes, eu entro num estado estranho: é como se eu estivesse me observando, estudando quais consequências aquele ato traz para mim. Fico me perguntando se sou a única pessoa desse mundo que faz isso…

Acho que um exemplo bom desse tipo de comportamento é o que tenho feito ultimamente. Às vezes, por algum motivo que me é estranho, leio coisas ruins escritas por pessoas extremamente insensatas. E, talvez pelo mesmo motivo misterioso, eu fico mal com o que leio, mesmo sabendo que, colocando na balança o que essas pessoas fazem e o que eu faço, a diferença é gigantesca. Então por que raios eu fico mal quando leio as besteiras que são praticamente vomitadas por essas pessoas?

Talvez algumas pessoas (eu incluso) tenham um radar pra sentimentos fortes. Por exemplo, um gesto de altruísmo é algo que consegue tocar o fundo da alma, e merece ser apreciado como um vinho raro. Mas, em contrapartida, uma expressão de raiva, desprezo ou incompreensão também capta a atenção de uma forma quase inevitável. O mistério que esse gesto, muitas vezes incoerente, esconde é algo que me deixa quase aficcionado, como se eu estivesse lendo um livro e não quisesse parar antes de chegar no final. Por que uma pessoa se coloca num papel por vezes ridículo, apenas por conta de uma opinião? Por que essa pessoa, na ânsia de criticar um comportamento, um pensamento, ou uma ideologia, muitas vezes exibe exatamente as mesmas características que repudia? O que faz um ser humano, cheio de falhas e limitações, subir num (muitas vezes falso) pedestal e esquecer que já esteve lá embaixo?

Felizmente, as questões acima, por mais intrigantes que sejam, não têm me prendido por muito tempo. Acho que, nesse processo de aprendizagem a que chamamos de “vida”, estou num ponto em que percebo claramente o caos que reina na cabeça dessas pessoas, e tento me afastar dele. Mas, mais importante que isso, acho que me dou conta de você pode escolher ser a mudança que quer ver no mundo (Gandhi), ou ficar ladrando enquanto a caravana passa… E eu definitivamente não quero perder meu tempo comparando códigos pra dizer quem é melhor.