<div dir="ltr"><div class="gmail_default" style="font-family:verdana,sans-serif">Hi Davide,</div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default" style="font-family:verdana,sans-serif">Thanks for the email and updating the notebook with the additions.</div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default" style="font-family:verdana,sans-serif">I was wondering if the list-at-large has any ideas where the inconsistencies come from re: number of emails filtered per year or month?</div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default" style="font-family:verdana,sans-serif">Because it throws up some real questions regarding using the data presented by the tool in academic publications.</div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default" style="font-family:verdana,sans-serif">Best,</div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default" style="font-family:verdana,sans-serif">Corinne </div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jul 25, 2018 at 2:03 PM, Beraldo, Davide <span dir="ltr"><<a href="mailto:d.beraldo@uva.nl" target="_blank">d.beraldo@uva.nl</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt">Hey Corinne,<br>
<br>
So I have integrated the first two additional measures (relative user activity and responsiveness -interesting measure btw)<br>
<br>
As for the third (you want to compute the yearly growth rate of users activity, if I got it right), I have been struggling with it a bit but haven't come to a working solution yet. Unfortunately (so to say ;P) I'll be traveling till Monday, so I can start to
work on it again next week. <br>
<br>
Also, I have tested the inconsistency you mention and I do get them on other mls as well. When measuring the number of emails as the length of the (filtered) archive, it gives a slightly different result then when filtering per month and doing the total. I
tried to dig into it but really cannot make sense of it at the moment<br>
<br>
I attach the notebook with the additions so you can test it locally if useful. I have integrated the new features in the 4th and 6th cells.<br>
<br>
I'll get back to you next week!<br>
<br>
Cheers,<br>
Davide<br>
<br>
<br>
<div style="font-family:Times New Roman;color:#000000;font-size:16px">
<hr>
<div id="m_7484682546014780444divRpF630692" style="direction:ltr"><font size="2" face="Tahoma" color="#000000"><b>From:</b> <a href="mailto:cattekwaad@gmail.com" target="_blank">cattekwaad@gmail.com</a> [<a href="mailto:cattekwaad@gmail.com" target="_blank">cattekwaad@gmail.com</a>] on behalf of Corinne Cath [<a href="mailto:corinnecath@gmail.com" target="_blank">corinnecath@gmail.com</a>]<br>
<b>Sent:</b> Tuesday, July 24, 2018 5:51 PM<br>
<b>To:</b> Beraldo, Davide<br>
<b>Cc:</b> <a href="mailto:bigbang-dev@data-activism.net" target="_blank">bigbang-dev@data-activism.net</a><br>
<b>Subject:</b> Re: [Bigbang-dev] Big Bang Notebook Questions Corinne<br>
</font><br>
</div><div><div class="h5">
<div></div>
<div>
<div dir="ltr">
<div class="gmail_default" style="font-family:verdana,sans-serif">Hi Davide,</div>
<div class="gmail_default" style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif">Many many thanks! So for the last one, I would like for each user how many emails they sent in each year and how those relate to each other. My phrasing is a bit clumsy, my apologies allow me
to demonstrate:</div>
<div class="gmail_default" style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif">So if Corinne sent 10 emails in year 1 and 20 in year two, that is a 50% increase of her emails. I guess I could also do that by head/calculator. </div>
<div class="gmail_default" style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif">I will also send a longer email to the list explaining some issues I ran into today!</div>
<div class="gmail_default" style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif">Best,</div>
<div class="gmail_default" style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif"><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Jul 24, 2018 at 5:24 PM, Beraldo, Davide <span dir="ltr">
<<a href="mailto:d.beraldo@uva.nl" rel="noopener noreferrer" target="_blank">d.beraldo@uva.nl</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt"><font size="3">Hi Corinne,
<br>
<br>
sorry I totally forgot to get back on this. Responding on coding-related questions.</font><br>
<br>
<font size="2"><br>
<span></span></font>
<div style="font-family:Times New Roman;color:#000000;font-size:16px">
<div>
<div dir="ltr">
<div class="gmail_default"><span>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"><i><font face="verdana, sans-serif"> In [6]: top senders over a time period<span></span></font></i></font></p>
<font size="2"></font>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2" face="verdana, sans-serif"> Currently, it gives the absolute numbers, for example, niels ten oever 77.0
<span></span></font></p>
<font size="2"></font>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font face="verdana, sans-serif"><font size="2"> It would be great to know what percentage that represents of the total number of emails sent in the period specified. So, those
77 emails are they 0.5, 5, 50% etc of the total over that period? </font><span></span></font></p>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font face="Tahoma"><span><br>
</span></font></p>
</span>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="3"><span><font face="Tahoma">Easy peasy</font><br>
</span></font></p>
<span>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"><span><font face="verdana, sans-serif"><br>
</font></span></font></p>
<font size="2"></font>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"><i><font face="verdana, sans-serif"> In [7]: number of emails in a time frame<span></span></font></i></font></p>
<font size="2"></font>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2" face="verdana, sans-serif"> I was wondering if it would also be possible to indicate the number of threads versus single emails (with no responses) to get a sense of
how responsive a mailing list is in a certain time period.<span></span></font></p>
<font size="2"></font>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"><span><font face="verdana, sans-serif"> </font></span></font></p>
</span><font face="verdana, sans-serif"><font face="Tahoma">Yes</font><br>
</font><span>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"> <font face="verdana, sans-serif"><br>
</font></p>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font face="verdana, sans-serif">
<font size="2">In [8]: I would be interested in, for instance, the average number of emails per user, across multiple years.</font></font></p>
<font size="2"></font>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"> <font face="verdana, sans-serif">It seems that currently, the numbers presented are not the average across the time specified but the absolute. That might also be because
for the test run, I set <span></span></font></font></p>
<font size="2"></font>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2">
</font><span><br>
</span></p>
<span><font face="verdana, sans-serif"><span><font face="verdana, sans-serif"> </font></span><font size="2">
</font></font></span>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"> <font face="verdana, sans-serif">date_from = pd.datetime(2014,10,1,tzinfo=p<wbr>ytz.utc)<span></span></font></font></p>
<font size="2"></font>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"> <font face="verdana, sans-serif">date_to = pd.datetime(2015,11,30,tzinfo=<wbr>pytz.utc)<span></span></font></font></p>
<font size="2"></font>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"> <font face="verdana, sans-serif">which is only a little over a year.
<br>
</font></font></p>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"><font face="verdana, sans-serif"><br>
</font></font></p>
</span>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"><font face="verdana, sans-serif"><font size="3"><font face="Tahoma">OK I think I misunderstood your question before. So you want to do: for each user, count how many emails
she sent in a timeframe, divide by number of years? If so, be aware that if you don't use a round number of years you would get inconsistent results on the tails. Also, do you mean year as in 365 days from date_from, or year as in 2015,2016,2017,...?</font></font></font></font></p>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2" face="Tahoma"><font size="3"><br>
</font></font></p>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"><font face="verdana, sans-serif"><font size="3"><font face="Tahoma">I'll try to incorporate these changes tomorrow, hope it's not too late!<br>
</font></font></font></font></p>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"><font face="verdana, sans-serif"><font size="3"><font face="Tahoma"><br>
</font></font></font></font></p>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"><font face="verdana, sans-serif"><font size="3"><font face="Tahoma"><br>
</font></font></font></font></p>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"><font face="verdana, sans-serif"><font size="3"><font face="Tahoma">Cheers,</font></font></font></font></p>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 0.0001pt"><font size="2"><font face="verdana, sans-serif"><font size="3"><font face="Tahoma">Davide<br>
</font></font></font></font></p>
<span></span></div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div class="m_7484682546014780444gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr"><span style="font-family:verdana,sans-serif">Corinne Cath <br>
Ph.D. Candidate, Oxford Internet Institute & Alan Turing Institute <br>
<br>
<span style="color:rgb(68,68,68)">Web: <a href="http://www.oii.ox.ac.uk/people/corinne-cath" rel="noopener noreferrer" target="_blank">
www.oii.ox.ac.uk/people/<wbr>corinne-cath</a> <br>
Email: <a href="mailto:ccath@turing.ac.uk" rel="noopener noreferrer" target="_blank">
ccath@turing.ac.uk</a> & <a href="mailto:corinnecath@gmail.com" rel="noopener noreferrer" target="_blank">
corinnecath@gmail.com</a><br>
Twitter: @C_Cath</span><br>
</span></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div></div>
</div>
</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><span style="font-family:verdana,sans-serif">Corinne Cath <br>Ph.D. Candidate, Oxford Internet Institute & Alan Turing Institute <br><br><span style="color:rgb(68,68,68)">Web: <a href="http://www.oii.ox.ac.uk/people/corinne-cath" target="_blank">www.oii.ox.ac.uk/people/corinne-cath</a> <br>Email: <a href="mailto:ccath@turing.ac.uk" target="_blank">ccath@turing.ac.uk</a> & <a href="mailto:corinnecath@gmail.com" target="_blank">corinnecath@gmail.com</a><br>Twitter: @C_Cath</span><br></span></div></div></div></div></div></div></div></div></div></div></div></div>
</div>