<div dir="ltr">Hi Niels & Riccardo,<div>the argument 'instant_dump' for the ListservArchive class object does not exist anymore in the up-to-date 'main' branch of the git repo.</div><div>@Niels: Do you mean that you did a 'git pull' and encountered the TypeError caused by missing 'instant_dump' too?<br></div><div><br></div><div>But as I said in another message, we are not quite there yet for 3GPP and IEEE to use the 'conventional' method on how BigBang scrapes archives such as W3C.</div><div>I attached a small examples that shows how you can currently scrape the 3GPP archive and save it to mbox files in the CONFIG.mail_path folder.</div><div>Be aware that this could take very long and could use a lot of memory.</div><div><br></div><div>Best Wishes,</div><div>Christoph</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Op do 22 apr. 2021 om 17:17 schreef Niels ten Oever <<a href="mailto:mail@nielstenoever.net">mail@nielstenoever.net</a>>:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Riccardo and Christoph,<br>
<br>
I see there might be an issue with the usage of special characters in the mailinglist URLs, to get it working I had to put a '\' in front on the '?', but this could also be fixed by using " " around the URL. However, after that fetching did not work either - so let's ask Christoph (cc).<br>
<br>
Cheers,<br>
<br>
Niels<br>
<br>
<br>
<br>
<br>
<br>
<br>
On 22-04-2021 17:43, Riccardo Nanni wrote:<br>
> Hi Niels,<br>
> <br>
> thanks for your answer!<br>
> I did, and I found the changes I can see in Github (e.g. the listserv.3GPP.txt file, etc.).<br>
> I did it again when I saw it didn't work and it says 'già aggiornato' (already updated).<br>
> <br>
> Riccardo<br>
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------<br>
> *Da:* Bigbang-user <<a href="mailto:bigbang-user-bounces@data-activism.net" target="_blank">bigbang-user-bounces@data-activism.net</a>> per conto di Niels ten Oever <<a href="mailto:mail@nielstenoever.net" target="_blank">mail@nielstenoever.net</a>><br>
> *Inviato:* giovedì 22 aprile 2021 17:38<br>
> *A:* <a href="mailto:bigbang-user@data-activism.net" target="_blank">bigbang-user@data-activism.net</a> <<a href="mailto:bigbang-user@data-activism.net" target="_blank">bigbang-user@data-activism.net</a>><br>
> *Oggetto:* Re: [Bigbang-user] Issue with listserv<br>
> <br>
> Hi Riccardo,<br>
> <br>
> This is not a very informed response - but did you first do:<br>
> <br>
> git pull<br>
> <br>
> to ensure that you have the latest version with all the recent changes?<br>
> <br>
> Best,<br>
> <br>
> Niels<br>
> <br>
> On 22-04-2021 17:31, Riccardo Nanni wrote:<br>
>> Dear all,<br>
>> <br>
>> how are you?<br>
>> I tried to collect email from 3GPP by running these commands:<br>
>> python bin/collect_mail.py -u <a href="https://list.etsi.org/scripts/wa.exe" rel="noreferrer" target="_blank">https://list.etsi.org/scripts/wa.exe</a>? <<a href="https://list.etsi.org/scripts/wa.exe" rel="noreferrer" target="_blank">https://list.etsi.org/scripts/wa.exe</a>?> <<a href="https://list.etsi.org/scripts/wa.exe" rel="noreferrer" target="_blank">https://list.etsi.org/scripts/wa.exe</a>? <<a href="https://list.etsi.org/scripts/wa.exe" rel="noreferrer" target="_blank">https://list.etsi.org/scripts/wa.exe</a>?>>;<br>
>> python3 bin/collect_mail.py -u <a href="https://list.etsi.org/scripts/wa.exe" rel="noreferrer" target="_blank">https://list.etsi.org/scripts/wa.exe</a>? <<a href="https://list.etsi.org/scripts/wa.exe" rel="noreferrer" target="_blank">https://list.etsi.org/scripts/wa.exe</a>?> <<a href="https://list.etsi.org/scripts/wa.exe" rel="noreferrer" target="_blank">https://list.etsi.org/scripts/wa.exe</a>? <<a href="https://list.etsi.org/scripts/wa.exe" rel="noreferrer" target="_blank">https://list.etsi.org/scripts/wa.exe</a>?>><br>
>> AND<br>
>> python3 bin/collect_mail.py -f examples/url_collections/listserv.3GPP.txt<br>
>> <br>
>> Also tried to scrape a specific group's list with the same commands: <a href="https://list.etsi.org/scripts/wa.exe?A0=3GPP_TSG_RAN" rel="noreferrer" target="_blank">https://list.etsi.org/scripts/wa.exe?A0=3GPP_TSG_RAN</a> <<a href="https://list.etsi.org/scripts/wa.exe?A0=3GPP_TSG_RAN" rel="noreferrer" target="_blank">https://list.etsi.org/scripts/wa.exe?A0=3GPP_TSG_RAN</a>> <<a href="https://list.etsi.org/scripts/wa.exe?A0=3GPP_TSG_RAN" rel="noreferrer" target="_blank">https://list.etsi.org/scripts/wa.exe?A0=3GPP_TSG_RAN</a> <<a href="https://list.etsi.org/scripts/wa.exe?A0=3GPP_TSG_RAN" rel="noreferrer" target="_blank">https://list.etsi.org/scripts/wa.exe?A0=3GPP_TSG_RAN</a>>><br>
>> <br>
>> I get the following error:<br>
>> TypeError: from_url() got an unexpected keyword argument 'instant_dump'<br>
>> <br>
>> I don't understand what I'm missing. Can you help me, please?<br>
>> Thanks a lot in advance! The only similar argument I could find on Stackoverflow has no answers...<br>
>> <br>
>> Riccardo<br>
>> <br>
>> <br>
>> <br>
>> <br>
>> _______________________________________________<br>
>> Bigbang-user mailing list<br>
>> <a href="mailto:Bigbang-user@data-activism.net" target="_blank">Bigbang-user@data-activism.net</a><br>
>> <a href="https://lists.ghserv.net/mailman/listinfo/bigbang-user" rel="noreferrer" target="_blank">https://lists.ghserv.net/mailman/listinfo/bigbang-user</a> <<a href="https://lists.ghserv.net/mailman/listinfo/bigbang-user" rel="noreferrer" target="_blank">https://lists.ghserv.net/mailman/listinfo/bigbang-user</a>><br>
>> <br>
> <br>
> -- <br>
> Niels ten Oever, PhD<br>
> Postdoctoral Researcher - Media Studies Department - University of Amsterdam<br>
> Research Fellow - Centre for Internet and Human Rights - European University Viadrina<br>
> Associated Scholar - Centro de Tecnologia e Sociedade - Fundação Getúlio Vargas<br>
> <br>
> <a href="https://nielstenoever.net" rel="noreferrer" target="_blank">https://nielstenoever.net</a> <<a href="https://nielstenoever.net" rel="noreferrer" target="_blank">https://nielstenoever.net</a>> - <a href="mailto:mail@nielstenoever.net" target="_blank">mail@nielstenoever.net</a> - @nielstenoever - +31629051853<br>
> PGP: 2458 0B70 5C4A FD8A 9488 643A 0ED8 3F3A 468A C8B3<br>
> <br>
> Read my latest article on Internet infrastructure governance in New Media & Society here: <a href="https://journals.sagepub.com/doi/full/10.1177/1461444820929320" rel="noreferrer" target="_blank">https://journals.sagepub.com/doi/full/10.1177/1461444820929320</a> <<a href="https://journals.sagepub.com/doi/full/10.1177/1461444820929320" rel="noreferrer" target="_blank">https://journals.sagepub.com/doi/full/10.1177/1461444820929320</a>><br>
> <br>
> _______________________________________________<br>
> Bigbang-user mailing list<br>
> <a href="mailto:Bigbang-user@data-activism.net" target="_blank">Bigbang-user@data-activism.net</a><br>
> <a href="https://lists.ghserv.net/mailman/listinfo/bigbang-user" rel="noreferrer" target="_blank">https://lists.ghserv.net/mailman/listinfo/bigbang-user</a> <<a href="https://lists.ghserv.net/mailman/listinfo/bigbang-user" rel="noreferrer" target="_blank">https://lists.ghserv.net/mailman/listinfo/bigbang-user</a>><br>
<br>
-- <br>
Niels ten Oever, PhD<br>
Postdoctoral Researcher - Media Studies Department - University of Amsterdam<br>
Research Fellow - Centre for Internet and Human Rights - European University Viadrina<br>
Associated Scholar - Centro de Tecnologia e Sociedade - Fundação Getúlio Vargas<br>
<br>
<a href="https://nielstenoever.net" rel="noreferrer" target="_blank">https://nielstenoever.net</a> - <a href="mailto:mail@nielstenoever.net" target="_blank">mail@nielstenoever.net</a> - @nielstenoever - +31629051853<br>
PGP: 2458 0B70 5C4A FD8A 9488 643A 0ED8 3F3A 468A C8B3<br>
<br>
Read my latest article on Internet infrastructure governance in New Media & Society here: <a href="https://journals.sagepub.com/doi/full/10.1177/1461444820929320" rel="noreferrer" target="_blank">https://journals.sagepub.com/doi/full/10.1177/1461444820929320</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><font size="3" face="Times New Roman,Times,serif" color="black"><font size="3" face="Times New Roman,Times,serif,EmojiFont,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe UI Symbol,Android Emoji,EmojiSymbols" color="black"><font size="1" face="Tahoma"><font size="1" face="Tahoma"><font size="1" face="Tahoma"><font size="1" face="Tahoma"><span style="font-size:13px"><font size="2" face="Times New Roman" color="#666699"><span style="background-color:rgb(255,255,255);color:rgb(0,111,201);font-size:9pt;line-height:normal"><><><><><><><><><><><><><><><><></span><span style="background-color:rgb(255,255,255);color:rgb(0,111,201)"><br></span></font><font color="#666699"><span style="background-color:rgb(255,255,255);color:rgb(0,111,201)"></span></font><span style="background-color:rgb(255,255,255);color:rgb(0,111,201)"></span></span><span style="font-size:9pt;line-height:normal"></span><span style="font-size:13px"></span></font></font></font></font></font></font><div><font size="3" face="Times New Roman,Times,serif" color="black"><font size="3" face="Times New Roman,Times,serif,EmojiFont,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe UI Symbol,Android Emoji,EmojiSymbols" color="black"><font size="1" face="Tahoma"><span style="font-size:13px"><span style="background-color:rgb(255,255,255);color:rgb(0,111,201)"></span></span></font></font></font><div><font size="3" face="Times New Roman,Times,serif" color="black"><font size="3" face="Times New Roman,Times,serif,EmojiFont,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe UI Symbol,Android Emoji,EmojiSymbols" color="black"><font size="2" face="Times New Roman" color="#666699"><i><font color="black"><font size="1"><font face="Tahoma"><span style="background-color:rgb(255,255,255);color:rgb(0,111,201);font-size:9pt;line-height:normal">Christoph Becker</span></font></font></font> <font size="3" face="Times New Roman,Times,serif" color="black"><font size="3" face="Times New Roman,Times,serif,EmojiFont,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe UI Symbol,Android Emoji,EmojiSymbols" color="black"><font size="1" face="Tahoma"><font size="1" face="Tahoma"><font size="1" face="Tahoma"><font size="1" face="Tahoma"><span><font size="1" face="Tahoma"><span><font size="2" face="Times New Roman" color="#666699"><i><font color="black"><font size="1"><font face="Tahoma"><span style="background-color:rgb(255,255,255);color:rgb(0,111,201);font-size:9pt;line-height:normal">(</span><span style="background-color:rgb(255,255,255);color:rgb(0,111,201);font-size:9pt;line-height:normal"><em>he/him/his</em></span><span style="background-color:rgb(255,255,255);color:rgb(0,111,201);font-size:9pt;line-height:normal">)</span></font></font></font></i></font></span></font></span></font></font></font></font></font></font></i></font></font></font><i><br><span style="background-color:rgb(255,255,255);color:rgb(0,111,201);font-size:9pt;line-height:normal">PhD at the<br></span></i></div><div><font size="2" face="Times New Roman" color="#666699"><i><span style="background-color:rgb(255,255,255);color:rgb(0,111,201);font-size:9pt;line-height:normal">Institute for Data Science and</span></i></font><br><span style="background-color:rgb(255,255,255);color:rgb(0,111,201)"></span></div><span style="font-size:9pt;line-height:normal"></span><span style="font-size:13px"><span style="background-color:rgb(255,255,255);color:rgb(0,111,201)"></span><span style="background-color:rgb(255,255,255);color:rgb(0,111,201)"></span><div><font size="2" face="Times New Roman" color="#666699"><i><span style="background-color:rgb(255,255,255);color:rgb(0,111,201);font-size:9pt;line-height:normal">Institute for Computational Cosmology</span></i></font></div><span style="background-color:rgb(255,255,255);color:rgb(0,111,201)"></span></span><span style="font-size:9pt;line-height:normal"></span><span style="font-size:13px"><div><font size="2" face="Times New Roman" color="#666699"><i><span style="background-color:rgb(255,255,255);color:rgb(0,111,201);font-size:9pt;line-height:normal">Durham University</span></i></font></div><span style="background-color:rgb(255,255,255);color:rgb(0,111,201)"></span></span><span style="font-size:9pt;line-height:normal"></span><span style="font-size:13px"><div><font size="1"><span style="font-size:13px"><font size="2" face="Times New Roman" color="#666699"><i><span style="background-color:rgb(255,255,255);color:rgb(0,111,201);font-size:9pt;line-height:normal">United Kingdom</span></i></font></span></font></div><div><a href="http://christovis.github.io" target="_blank"><font size="1"><span style="font-size:13px"><font size="2" face="Times New Roman" color="#666699"><i><span style="background-color:rgb(255,255,255);color:rgb(0,111,201)"><font size="3" face="Times New Roman,Times,serif" color="black"><font size="3" face="Times New Roman,Times,serif,EmojiFont,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe UI Symbol,Android Emoji,EmojiSymbols" color="black"><font size="1" face="Tahoma"><font size="1" face="Tahoma"><font size="1" face="Tahoma"><font size="1" face="Tahoma"><span><font size="1" face="Tahoma"><span><font size="2" face="Times New Roman" color="#666699"><i><span style="background-color:rgb(255,255,255);color:rgb(0,111,201);font-size:9pt;line-height:normal">christovis.github.io</span></i></font></span></font></span></font></font></font></font></font></font></span></i></font></span></font></a></div></span></div></div></div>