Arrfab's blog - armhfphttps://arrfab.net/2018-01-10T00:00:00+01:00Some tips and tricks, mostly around CentOSUsing a RaspberryPI3 as Unifi AP controller with CentOS 72018-01-10T00:00:00+01:002018-01-10T00:00:00+01:00Fabian Arrotintag:arrfab.net,2018-01-10:/posts/2018/Jan/10/using-a-raspberrypi3-as-unifi-ap-controller-with-centos-7/<p>That's something I should have blogged about earlier, but I almost forgot about it, until I read on twitter other people having replaced their home network equipment with Ubnt/Ubiquiti gear so I realized that it was on my to 'TOBLOG' list.</p>
<p>During the winter holidays, the whole family was at home, and also with kids on the WiFi network. Of course I already had a different wlan for them, separated/seggregated from the main one, but plenty of things weren't really working on that crappy device. So it was time to setup something else. I had opportunity to play with some <a href="https://www.ubnt.com/">Ubiquiti</a> devices in the past, so finding even an old <a href="https://www.ubnt.com/unifi/unifi-ap/">Unifi UAP model</a> was enough for my needs (just need Access Point, routing/firewall being done on something else).</p>
<p>If you've already played with those tools, you know that you need a controller to setup the devices up , and because it's 'only' a java/mongodb stack, I thought it would be trivial to setup on a low-end device like <a href="https://www.raspberrypi.org/">RaspberryPi3</a> (not limited to that , so all armhfp boards on which you can run CentOS would work)</p>
<p>After having installed <a href="http://mirror.centos.org/altarch/7/isos/armhfp/">CentOS 7 armhfp minimal</a> on the device, and once …</p><p>That's something I should have blogged about earlier, but I almost forgot about it, until I read on twitter other people having replaced their home network equipment with Ubnt/Ubiquiti gear so I realized that it was on my to 'TOBLOG' list.</p>
<p>During the winter holidays, the whole family was at home, and also with kids on the WiFi network. Of course I already had a different wlan for them, separated/seggregated from the main one, but plenty of things weren't really working on that crappy device. So it was time to setup something else. I had opportunity to play with some <a href="https://www.ubnt.com/">Ubiquiti</a> devices in the past, so finding even an old <a href="https://www.ubnt.com/unifi/unifi-ap/">Unifi UAP model</a> was enough for my needs (just need Access Point, routing/firewall being done on something else).</p>
<p>If you've already played with those tools, you know that you need a controller to setup the devices up , and because it's 'only' a java/mongodb stack, I thought it would be trivial to setup on a low-end device like <a href="https://www.raspberrypi.org/">RaspberryPi3</a> (not limited to that , so all armhfp boards on which you can run CentOS would work)</p>
<p>After having installed <a href="http://mirror.centos.org/altarch/7/isos/armhfp/">CentOS 7 armhfp minimal</a> on the device, and once logged, I just had to add the mandatory <a href="https://wiki.centos.org/SpecialInterestGroup/AltArch/Arm32#head-f2a772703b3caa90cc284e01bc87423ce9a87bcd">unofficial epel repository</a> for mongodb </p>
<div class="highlight"><pre><span></span><span class="nv">cat</span> <span class="o">></span> <span class="o">/</span><span class="nv">etc</span><span class="o">/</span><span class="nv">yum</span>.<span class="nv">repos</span>.<span class="nv">d</span><span class="o">/</span><span class="nv">epel</span>.<span class="nv">repo</span> <span class="o"><<</span> <span class="nv">EOF</span>
[<span class="nv">epel</span>]
<span class="nv">name</span><span class="o">=</span><span class="nv">Epel</span> <span class="nv">rebuild</span> <span class="k">for</span> <span class="nv">armhfp</span>
<span class="nv">baseurl</span><span class="o">=</span><span class="nv">https</span>:<span class="o">//</span><span class="nv">armv7</span>.<span class="nv">dev</span>.<span class="nv">centos</span>.<span class="nv">org</span><span class="o">/</span><span class="nv">repodir</span><span class="o">/</span><span class="nv">epel</span><span class="o">-</span><span class="nv">pass</span><span class="o">-</span><span class="mi">1</span><span class="o">/</span>
<span class="nv">enabled</span><span class="o">=</span><span class="mi">1</span>
<span class="nv">gpgcheck</span><span class="o">=</span><span class="mi">0</span>
<span class="nv">EOF</span>
</pre></div>
<p>After that, just installed what's required to run the application :</p>
<div class="highlight"><pre><span></span><span class="n">yum</span> <span class="n">install</span> <span class="n">mongodb</span> <span class="n">mongodb</span><span class="o">-</span><span class="n">server</span> <span class="n">java</span><span class="o">-</span><span class="mi">1</span><span class="p">.</span><span class="mi">8</span><span class="p">.</span><span class="mi">0</span><span class="o">-</span><span class="n">openjdk</span><span class="o">-</span><span class="n">headless</span> <span class="o">-</span><span class="n">y</span>
</pre></div>
<p>The "interesting" part is that now Ubnt only provides .deb packages , so we just have to download/extract what we need (it's all java code) and start it : </p>
<div class="highlight"><pre><span></span><span class="n">tmp_dir</span><span class="o">=</span><span class="err">$</span><span class="p">(</span><span class="n">mktemp</span> <span class="o">-</span><span class="n">d</span><span class="p">)</span>
<span class="n">cd</span> <span class="err">$</span><span class="n">tmp_dir</span>
<span class="n">curl</span> <span class="o">-</span><span class="n">O</span> <span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">dl</span><span class="p">.</span><span class="n">ubnt</span><span class="p">.</span><span class="n">com</span><span class="o">/</span><span class="n">unifi</span><span class="o">/</span><span class="mi">5</span><span class="p">.</span><span class="mi">6</span><span class="p">.</span><span class="mi">26</span><span class="o">/</span><span class="n">unifi_sysvinit_all</span><span class="p">.</span><span class="n">deb</span>
<span class="n">ar</span> <span class="n">vx</span> <span class="n">unifi_sysvinit_all</span><span class="p">.</span><span class="n">deb</span>
<span class="n">tar</span> <span class="n">xvf</span> <span class="k">data</span><span class="p">.</span><span class="n">tar</span><span class="p">.</span><span class="n">xz</span>
<span class="n">mv</span> <span class="n">usr</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">unifi</span><span class="o">/</span> <span class="o">/</span><span class="n">opt</span><span class="o">/</span><span class="n">UniFi</span>
<span class="n">cd</span> <span class="o">/</span><span class="n">opt</span><span class="o">/</span><span class="n">UniFi</span><span class="o">/</span><span class="n">bin</span>
<span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">rm</span> <span class="o">-</span><span class="n">Rf</span> <span class="err">$</span><span class="n">tmp_dir</span>
<span class="n">ln</span> <span class="o">-</span><span class="n">s</span> <span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">mongod</span>
</pre></div>
<p>You can start it "by hand" but let's create a simple systemd file and use it directly :</p>
<div class="highlight"><pre><span></span><span class="n">cat</span><span class="w"> </span><span class="o">></span><span class="w"> </span><span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">systemd</span><span class="o">/</span><span class="k">system</span><span class="o">/</span><span class="n">unifi</span><span class="p">.</span><span class="n">service</span><span class="w"> </span><span class="o"><<</span><span class="w"> </span><span class="n">EOF</span><span class="w"></span>
<span class="o">[</span><span class="n">Unit</span><span class="o">]</span><span class="w"></span>
<span class="n">Description</span><span class="o">=</span><span class="n">UBNT</span><span class="w"> </span><span class="n">UniFi</span><span class="w"> </span><span class="n">Controller</span><span class="w"></span>
<span class="k">After</span><span class="o">=</span><span class="n">syslog</span><span class="p">.</span><span class="n">target</span><span class="w"> </span><span class="n">network</span><span class="p">.</span><span class="n">target</span><span class="w"></span>
<span class="o">[</span><span class="n">Service</span><span class="o">]</span><span class="w"></span>
<span class="n">WorkingDirectory</span><span class="o">=/</span><span class="n">opt</span><span class="o">/</span><span class="n">UniFi</span><span class="w"></span>
<span class="n">ExecStart</span><span class="o">=/</span><span class="n">usr</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">java</span><span class="w"> </span><span class="o">-</span><span class="n">jar</span><span class="w"> </span><span class="o">/</span><span class="n">opt</span><span class="o">/</span><span class="n">UniFi</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">ace</span><span class="p">.</span><span class="n">jar</span><span class="w"> </span><span class="k">start</span><span class="w"></span>
<span class="n">ExecStop</span><span class="o">=/</span><span class="n">usr</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">java</span><span class="w"> </span><span class="o">-</span><span class="n">jar</span><span class="w"> </span><span class="o">/</span><span class="n">opt</span><span class="o">/</span><span class="n">UniFi</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">ace</span><span class="p">.</span><span class="n">jar</span><span class="w"> </span><span class="n">stop</span><span class="w"></span>
<span class="o">[</span><span class="n">Install</span><span class="o">]</span><span class="w"></span>
<span class="n">WantedBy</span><span class="o">=</span><span class="n">multi</span><span class="o">-</span><span class="k">user</span><span class="p">.</span><span class="n">target</span><span class="w"></span>
<span class="n">EOF</span><span class="w"></span>
<span class="n">systemctl</span><span class="w"> </span><span class="n">daemon</span><span class="o">-</span><span class="n">reload</span><span class="w"></span>
<span class="n">systemctl</span><span class="w"> </span><span class="n">enable</span><span class="w"> </span><span class="n">unifi</span><span class="w"> </span><span class="c1">--now</span>
</pre></div>
<p>Don't forget that :</p>
<ul>
<li>it's "Java"</li>
<li>running on slow armhfp processor</li>
</ul>
<p>So that will take time to initialize. You can follow progress in /opt/UniFi/logs/server.log and wait for the TLS port to be opened : </p>
<div class="highlight"><pre><span></span><span class="k">while</span> <span class="nv">true</span> <span class="c1">; do sleep 1 ; ss -tanp|grep 8443 && break ; done</span>
</pre></div>
<p>Dont forget to open the needed ports for firewall and you can then reach the Unifi controller running on your armhfp board.</p>Using CentOS 7 armhfp VM on CentOS 7 aarch642017-09-29T00:00:00+02:002017-09-29T00:00:00+02:00Fabian Arrotintag:arrfab.net,2017-09-29:/posts/2017/Sep/29/using-centos-7-armhfp-vm-on-centos-7-aarch64/<p>Recently we got our hands on some aarch64 (aka ARMv8 / 64Bits) nodes running in a remote DC. On my (already too long) TODO/TOTEST list I had the idea of testing armhfp VM on top of aarch64. Reason is that when I need to test our packages, using my own <a href="https://www.cubietruck.com/">Cubietruck</a> or <a href="https://www.raspberrypi.org/">RaspberryPi3</a> is time consuming : removing the sdcard, reflashing with the correct <a href="http://mirror.centos.org/altarch/7/isos/armhfp/">CentOS 7 image</a> and booting/testing the pkg/update/etc ...</p>
<p>So is that possible to just automate this through available aarch64 node as hypervisor ? Sure ! and it's just pretty straightforward if you have already played with libvirt.
Let's so start with a CentOS 7 aarch64 minimal setup and then : </p>
<div class="highlight"><pre><span></span><span class="n">yum</span> <span class="n">install</span> <span class="n">qemu</span><span class="o">-</span><span class="n">kvm</span><span class="o">-</span><span class="n">tools</span> <span class="n">qemu</span><span class="o">-</span><span class="n">kvm</span> <span class="n">virt</span><span class="o">-</span><span class="n">install</span> <span class="n">libvirt</span> <span class="n">libvirt</span><span class="o">-</span><span class="n">python</span> <span class="n">libguestfs</span><span class="o">-</span><span class="n">tools</span><span class="o">-</span><span class="k">c</span>
<span class="n">systemctl</span> <span class="n">enable</span> <span class="n">libvirtd</span> <span class="c1">--now</span>
</pre></div>
<p>That's pretty basic but for armhfp we'll have to do some extra steps : qemu normally tries to simulate a bios/uefi boot, which armhfp doesn't support, and qemu doesn't emulate the mandatory uboot to just chainload to the RootFS from the guest VM.</p>
<p>So here is just what we need : </p>
<ul>
<li>Import the RootFS from an existing image</li>
</ul>
<div class="highlight"><pre><span></span><span class="n">curl</span> <span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">mirror</span><span class="p">.</span><span class="n">centos</span><span class="p">.</span><span class="n">org</span><span class="o">/</span><span class="n">altarch</span><span class="o">/</span><span class="mi">7</span><span class="o">/</span><span class="n">isos</span><span class="o">/</span><span class="n">armhfp</span><span class="o">/</span><span class="n">CentOS</span><span class="o">-</span><span class="n">Userland</span><span class="o">-</span><span class="mi">7 …</span></pre></div><p>Recently we got our hands on some aarch64 (aka ARMv8 / 64Bits) nodes running in a remote DC. On my (already too long) TODO/TOTEST list I had the idea of testing armhfp VM on top of aarch64. Reason is that when I need to test our packages, using my own <a href="https://www.cubietruck.com/">Cubietruck</a> or <a href="https://www.raspberrypi.org/">RaspberryPi3</a> is time consuming : removing the sdcard, reflashing with the correct <a href="http://mirror.centos.org/altarch/7/isos/armhfp/">CentOS 7 image</a> and booting/testing the pkg/update/etc ...</p>
<p>So is that possible to just automate this through available aarch64 node as hypervisor ? Sure ! and it's just pretty straightforward if you have already played with libvirt.
Let's so start with a CentOS 7 aarch64 minimal setup and then : </p>
<div class="highlight"><pre><span></span><span class="n">yum</span> <span class="n">install</span> <span class="n">qemu</span><span class="o">-</span><span class="n">kvm</span><span class="o">-</span><span class="n">tools</span> <span class="n">qemu</span><span class="o">-</span><span class="n">kvm</span> <span class="n">virt</span><span class="o">-</span><span class="n">install</span> <span class="n">libvirt</span> <span class="n">libvirt</span><span class="o">-</span><span class="n">python</span> <span class="n">libguestfs</span><span class="o">-</span><span class="n">tools</span><span class="o">-</span><span class="k">c</span>
<span class="n">systemctl</span> <span class="n">enable</span> <span class="n">libvirtd</span> <span class="c1">--now</span>
</pre></div>
<p>That's pretty basic but for armhfp we'll have to do some extra steps : qemu normally tries to simulate a bios/uefi boot, which armhfp doesn't support, and qemu doesn't emulate the mandatory uboot to just chainload to the RootFS from the guest VM.</p>
<p>So here is just what we need : </p>
<ul>
<li>Import the RootFS from an existing image</li>
</ul>
<div class="highlight"><pre><span></span><span class="n">curl</span> <span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">mirror</span><span class="p">.</span><span class="n">centos</span><span class="p">.</span><span class="n">org</span><span class="o">/</span><span class="n">altarch</span><span class="o">/</span><span class="mi">7</span><span class="o">/</span><span class="n">isos</span><span class="o">/</span><span class="n">armhfp</span><span class="o">/</span><span class="n">CentOS</span><span class="o">-</span><span class="n">Userland</span><span class="o">-</span><span class="mi">7</span><span class="o">-</span><span class="n">armv7hl</span><span class="o">-</span><span class="n">Minimal</span><span class="o">-</span><span class="mi">1708</span><span class="o">-</span><span class="n">CubieTruck</span><span class="p">.</span><span class="n">img</span><span class="p">.</span><span class="n">xz</span><span class="o">|</span><span class="n">unxz</span> <span class="o">>/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">libvirt</span><span class="o">/</span><span class="n">images</span><span class="o">/</span><span class="n">CentOS</span><span class="o">-</span><span class="n">Userland</span><span class="o">-</span><span class="mi">7</span><span class="o">-</span><span class="n">armv7hl</span><span class="o">-</span><span class="n">Minimal</span><span class="o">-</span><span class="mi">1708</span><span class="o">-</span><span class="n">CubieTruck</span><span class="p">.</span><span class="n">img</span>
</pre></div>
<ul>
<li>Convert image to <a href="https://en.wikipedia.org/wiki/Qcow">qcow2</a> (that will give us more flexibility) and extend it a little bit</li>
</ul>
<div class="highlight"><pre><span></span><span class="n">qemu</span><span class="o">-</span><span class="n">img</span> <span class="k">convert</span> <span class="o">-</span><span class="n">f</span> <span class="n">raw</span> <span class="o">-</span><span class="n">O</span> <span class="n">qcow2</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">libvirt</span><span class="o">/</span><span class="n">images</span><span class="o">/</span><span class="n">CentOS</span><span class="o">-</span><span class="n">Userland</span><span class="o">-</span><span class="mi">7</span><span class="o">-</span><span class="n">armv7hl</span><span class="o">-</span><span class="n">Minimal</span><span class="o">-</span><span class="mi">1708</span><span class="o">-</span><span class="n">CubieTruck</span><span class="p">.</span><span class="n">img</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">libvirt</span><span class="o">/</span><span class="n">images</span><span class="o">/</span><span class="n">CentOS</span><span class="o">-</span><span class="n">Userland</span><span class="o">-</span><span class="mi">7</span><span class="o">-</span><span class="n">armv7hl</span><span class="o">-</span><span class="n">Minimal</span><span class="o">-</span><span class="mi">1708</span><span class="o">-</span><span class="n">guest</span><span class="p">.</span><span class="n">qcow2</span>
<span class="n">qemu</span><span class="o">-</span><span class="n">img</span> <span class="n">resize</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">libvirt</span><span class="o">/</span><span class="n">images</span><span class="o">/</span><span class="n">CentOS</span><span class="o">-</span><span class="n">Userland</span><span class="o">-</span><span class="mi">7</span><span class="o">-</span><span class="n">armv7hl</span><span class="o">-</span><span class="n">Minimal</span><span class="o">-</span><span class="mi">1708</span><span class="o">-</span><span class="n">guest</span><span class="p">.</span><span class="n">qcow2</span> <span class="o">+</span><span class="mi">15</span><span class="k">G</span>
</pre></div>
<ul>
<li>Extract kernel+initrd as libvirt will boot that directly for the VM</li>
</ul>
<div class="highlight"><pre><span></span><span class="n">mkdir</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">libvirt</span><span class="o">/</span><span class="n">armhfp</span><span class="o">-</span><span class="n">boot</span>
<span class="n">virt</span><span class="o">-</span><span class="k">copy</span><span class="o">-</span><span class="k">out</span> <span class="o">-</span><span class="n">a</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">libvirt</span><span class="o">/</span><span class="n">images</span><span class="o">/</span><span class="n">CentOS</span><span class="o">-</span><span class="n">Userland</span><span class="o">-</span><span class="mi">7</span><span class="o">-</span><span class="n">armv7hl</span><span class="o">-</span><span class="n">Minimal</span><span class="o">-</span><span class="mi">1708</span><span class="o">-</span><span class="n">guest</span><span class="p">.</span><span class="n">qcow2</span> <span class="o">/</span><span class="n">boot</span><span class="o">/</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">libvirt</span><span class="o">/</span><span class="n">armhfp</span><span class="o">-</span><span class="n">boot</span><span class="o">/</span>
</pre></div>
<p>So now that we have a RootFS, and also kernel/initrd, we can just use virt-install to create the VM (pointing to existing backend qcow2) :</p>
<div class="highlight"><pre><span></span><span class="n">virt</span><span class="o">-</span><span class="n">install</span> \
<span class="o">--</span><span class="n">name</span> <span class="n">centos7_armhfp</span> \
<span class="o">--</span><span class="n">memory</span> <span class="mi">4096</span> \
<span class="o">--</span><span class="n">boot</span> <span class="n">kernel</span><span class="o">=/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">libvirt</span><span class="o">/</span><span class="n">armhfp</span><span class="o">-</span><span class="n">boot</span><span class="o">/</span><span class="n">boot</span><span class="o">/</span><span class="n">vmlinuz</span><span class="o">-</span><span class="mf">4.9</span><span class="o">.</span><span class="mi">40</span><span class="o">-</span><span class="mf">203.</span><span class="n">el7</span><span class="o">.</span><span class="n">armv7hl</span><span class="p">,</span><span class="n">initrd</span><span class="o">=/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">libvirt</span><span class="o">/</span><span class="n">armhfp</span><span class="o">-</span><span class="n">boot</span><span class="o">/</span><span class="n">boot</span><span class="o">/</span><span class="n">initramfs</span><span class="o">-</span><span class="mf">4.9</span><span class="o">.</span><span class="mi">40</span><span class="o">-</span><span class="mf">203.</span><span class="n">el7</span><span class="o">.</span><span class="n">armv7hl</span><span class="o">.</span><span class="n">img</span><span class="p">,</span><span class="n">kernel_args</span><span class="o">=</span><span class="s2">"console=ttyAMA0 rw root=/dev/sda3"</span> \
<span class="o">--</span><span class="n">disk</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">libvirt</span><span class="o">/</span><span class="n">images</span><span class="o">/</span><span class="n">CentOS</span><span class="o">-</span><span class="n">Userland</span><span class="o">-</span><span class="mi">7</span><span class="o">-</span><span class="n">armv7hl</span><span class="o">-</span><span class="n">Minimal</span><span class="o">-</span><span class="mi">1708</span><span class="o">-</span><span class="n">guest</span><span class="o">.</span><span class="n">qcow2</span> \
<span class="o">--</span><span class="kn">import</span> \
<span class="o">--</span><span class="n">arch</span> <span class="n">armv7l</span> \
<span class="o">--</span><span class="n">machine</span> <span class="n">virt</span> \
</pre></div>
<p>And here we go : we have a armhfp VM that boots <em>really</em> fast (compared to a armhfp board using a microsd card of course)</p>
<p>At this stage, you can configure the node, etc.. The only thing you have to remember is that of course kernel will be provided from <em>outside</em> the VM, so just extract it from an updated VM to boot on that kernel. Let's show how to do that, as in the above example, we configured the VM to run with 4Gb of ram, but only 3 are really seen inside (remember the 32bits mode and so the need for <a href="https://en.wikipedia.org/wiki/Physical_Address_Extension">PAE</a> on i386 ?)</p>
<p>So let's use this example to show how to switch kernel : From the armhfp VM : </p>
<div class="highlight"><pre><span></span># <span class="nv">Let</span> <span class="nv">extend</span> <span class="nv">first</span> <span class="nv">as</span> <span class="nv">we</span> <span class="nv">have</span> <span class="nv">bigger</span> <span class="nv">disk</span>
<span class="nv">growpart</span> <span class="o">/</span><span class="nv">dev</span><span class="o">/</span><span class="nv">sda</span> <span class="mi">3</span>
<span class="nv">resize2fs</span> <span class="o">/</span><span class="nv">dev</span><span class="o">/</span><span class="nv">sda3</span>
<span class="nv">yum</span> <span class="nv">update</span> <span class="o">-</span><span class="nv">y</span>
<span class="nv">yum</span> <span class="nv">install</span> <span class="nv">kernel</span><span class="o">-</span><span class="nv">lpae</span>
<span class="nv">systemctl</span> <span class="nv">poweroff</span> # <span class="nv">we</span><span class="s1">'</span><span class="s">ll modify libvirt conf file for new kernel</span>
</pre></div>
<p>Back to the hypervisor we can again extract needed files :</p>
<div class="highlight"><pre><span></span><span class="n">virt</span><span class="o">-</span><span class="k">copy</span><span class="o">-</span><span class="k">out</span> <span class="o">-</span><span class="n">a</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">libvirt</span><span class="o">/</span><span class="n">images</span><span class="o">/</span><span class="n">CentOS</span><span class="o">-</span><span class="n">Userland</span><span class="o">-</span><span class="mi">7</span><span class="o">-</span><span class="n">armv7hl</span><span class="o">-</span><span class="n">Minimal</span><span class="o">-</span><span class="mi">1708</span><span class="o">-</span><span class="n">guest</span><span class="p">.</span><span class="n">qcow2</span> <span class="o">/</span><span class="n">boot</span><span class="o">/</span><span class="n">vmlinuz</span><span class="o">-</span><span class="mi">4</span><span class="p">.</span><span class="mi">9</span><span class="p">.</span><span class="mi">50</span><span class="o">-</span><span class="mi">203</span><span class="p">.</span><span class="n">el7</span><span class="p">.</span><span class="n">armv7hl</span><span class="o">+</span><span class="n">lpae</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">libvirt</span><span class="o">/</span><span class="n">armhfp</span><span class="o">-</span><span class="n">boot</span><span class="o">/</span><span class="n">boot</span><span class="o">/</span>
<span class="n">virt</span><span class="o">-</span><span class="k">copy</span><span class="o">-</span><span class="k">out</span> <span class="o">-</span><span class="n">a</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">libvirt</span><span class="o">/</span><span class="n">images</span><span class="o">/</span><span class="n">CentOS</span><span class="o">-</span><span class="n">Userland</span><span class="o">-</span><span class="mi">7</span><span class="o">-</span><span class="n">armv7hl</span><span class="o">-</span><span class="n">Minimal</span><span class="o">-</span><span class="mi">1708</span><span class="o">-</span><span class="n">guest</span><span class="p">.</span><span class="n">qcow2</span> <span class="o">/</span><span class="n">boot</span><span class="o">/</span><span class="n">initramfs</span><span class="o">-</span><span class="mi">4</span><span class="p">.</span><span class="mi">9</span><span class="p">.</span><span class="mi">50</span><span class="o">-</span><span class="mi">203</span><span class="p">.</span><span class="n">el7</span><span class="p">.</span><span class="n">armv7hl</span><span class="o">+</span><span class="n">lpae</span><span class="p">.</span><span class="n">img</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">libvirt</span><span class="o">/</span><span class="n">armhfp</span><span class="o">-</span><span class="n">boot</span><span class="o">/</span><span class="n">boot</span><span class="o">/</span>
</pre></div>
<p>And just <code>virsh edit centos7_armhfp</code> so that kernel and armhfp are pointing to correct location:</p>
<div class="highlight"><pre><span></span><span class="nt"><kernel></span>/var/lib/libvirt/armhfp-boot/boot/vmlinuz-4.9.50-203.el7.armv7hl+lpae<span class="nt"></kernel></span>
<span class="nt"><initrd></span>/var/lib/libvirt/armhfp-boot/boot/initramfs-4.9.50-203.el7.armv7hl+lpae.img<span class="nt"></initrd></span>
</pre></div>
<p>Now that we have a "gold" image, we can even use exiting tools to provision quickly other nodes on that hypervisor ! :</p>
<div class="highlight"><pre><span></span><span class="n">time</span> <span class="n">virt</span><span class="o">-</span><span class="n">clone</span> <span class="c1">--original centos7_armhfp --name armhfp_guest1 --file /var/lib/libvirt/images/armhfp_guest1.qcow2</span>
<span class="n">Allocating</span> <span class="s1">'armhfp_guest1.qcow2'</span> <span class="o">|</span> <span class="mi">18</span> <span class="n">GB</span> <span class="mi">00</span><span class="p">:</span><span class="mi">00</span><span class="p">:</span><span class="mi">02</span>
<span class="n">Clone</span> <span class="s1">'armhfp_guest1'</span> <span class="n">created</span> <span class="n">successfully</span><span class="p">.</span>
<span class="nb">real</span> <span class="mi">0</span><span class="n">m2</span><span class="p">.</span><span class="mi">809</span><span class="n">s</span>
<span class="k">user</span> <span class="mi">0</span><span class="n">m0</span><span class="p">.</span><span class="mi">473</span><span class="n">s</span>
<span class="n">sys</span> <span class="mi">0</span><span class="n">m0</span><span class="p">.</span><span class="mi">062</span><span class="n">s</span>
<span class="n">time</span> <span class="n">virt</span><span class="o">-</span><span class="n">sysprep</span> <span class="c1">--add /var/lib/libvirt/images/armhfp_guest1.qcow2 --operations defaults,net-hwaddr,machine-id,net-hostname,ssh-hostkeys,udev-persistent-net --hostname guest1</span>
<span class="n">virsh</span> <span class="k">start</span> <span class="n">armhfp_guest1</span>
</pre></div>
<p>As simple as that.
Of course, in the previous example we were just using the default network from libvirt, and not any bridge, but you get the idea : all the rest with well-known concept for libvirt on linux.</p>