2 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
3 <html><head><title>Python: module telemetry.page.cloud_storage</title>
4 </head><body bgcolor="#f0f0f8">
6 <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="heading">
8 <td valign=bottom> <br>
9 <font color="#ffffff" face="helvetica, arial"> <br><big><big><strong><a href="telemetry.html"><font color="#ffffff">telemetry</font></a>.<a href="telemetry.page.html"><font color="#ffffff">page</font></a>.cloud_storage</strong></big></big></font></td
10 ><td align=right valign=bottom
11 ><font color="#ffffff" face="helvetica, arial"><a href=".">index</a><br><a href="../telemetry/page/cloud_storage.py">telemetry/page/cloud_storage.py</a></font></td></tr></table>
12 <p><tt>Wrappers for gsutil, for basic interaction with Google Cloud Storage.</tt></p>
14 <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section">
15 <tr bgcolor="#aa55cc">
16 <td colspan=3 valign=bottom> <br>
17 <font color="#ffffff" face="helvetica, arial"><big><strong>Modules</strong></big></font></td></tr>
19 <tr><td bgcolor="#aa55cc"><tt> </tt></td><td> </td>
20 <td width="100%"><table width="100%" summary="list"><tr><td width="25%" valign=top><a href="cStringIO.html">cStringIO</a><br>
21 <a href="hashlib.html">hashlib</a><br>
22 <a href="logging.html">logging</a><br>
23 </td><td width="25%" valign=top><a href="os.html">os</a><br>
24 <a href="subprocess.html">subprocess</a><br>
25 <a href="sys.html">sys</a><br>
26 </td><td width="25%" valign=top><a href="tarfile.html">tarfile</a><br>
27 <a href="urllib2.html">urllib2</a><br>
28 <a href="telemetry.core.util.html">telemetry.core.util</a><br>
29 </td><td width="25%" valign=top></td></tr></table></td></tr></table><p>
30 <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section">
31 <tr bgcolor="#ee77aa">
32 <td colspan=3 valign=bottom> <br>
33 <font color="#ffffff" face="helvetica, arial"><big><strong>Classes</strong></big></font></td></tr>
35 <tr><td bgcolor="#ee77aa"><tt> </tt></td><td> </td>
37 <dt><font face="helvetica, arial"><a href="exceptions.html#Exception">exceptions.Exception</a>(<a href="exceptions.html#BaseException">exceptions.BaseException</a>)
40 <dt><font face="helvetica, arial"><a href="telemetry.page.cloud_storage.html#CloudStorageError">CloudStorageError</a>
43 <dt><font face="helvetica, arial"><a href="telemetry.page.cloud_storage.html#CredentialsError">CredentialsError</a>
44 </font></dt><dt><font face="helvetica, arial"><a href="telemetry.page.cloud_storage.html#NotFoundError">NotFoundError</a>
45 </font></dt><dt><font face="helvetica, arial"><a href="telemetry.page.cloud_storage.html#PermissionError">PermissionError</a>
52 <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section">
53 <tr bgcolor="#ffc8d8">
54 <td colspan=3 valign=bottom> <br>
55 <font color="#000000" face="helvetica, arial"><a name="CloudStorageError">class <strong>CloudStorageError</strong></a>(<a href="exceptions.html#Exception">exceptions.Exception</a>)</font></td></tr>
57 <tr><td bgcolor="#ffc8d8"><tt> </tt></td><td> </td>
58 <td width="100%"><dl><dt>Method resolution order:</dt>
59 <dd><a href="telemetry.page.cloud_storage.html#CloudStorageError">CloudStorageError</a></dd>
60 <dd><a href="exceptions.html#Exception">exceptions.Exception</a></dd>
61 <dd><a href="exceptions.html#BaseException">exceptions.BaseException</a></dd>
62 <dd><a href="__builtin__.html#object">__builtin__.object</a></dd>
65 Data descriptors defined here:<br>
66 <dl><dt><strong>__weakref__</strong></dt>
67 <dd><tt>list of weak references to the object (if defined)</tt></dd>
70 Methods inherited from <a href="exceptions.html#Exception">exceptions.Exception</a>:<br>
71 <dl><dt><a name="CloudStorageError-__init__"><strong>__init__</strong></a>(...)</dt><dd><tt>x.<a href="#CloudStorageError-__init__">__init__</a>(...) initializes x; see help(type(x)) for signature</tt></dd></dl>
74 Data and other attributes inherited from <a href="exceptions.html#Exception">exceptions.Exception</a>:<br>
75 <dl><dt><strong>__new__</strong> = <built-in method __new__ of type object><dd><tt>T.<a href="#CloudStorageError-__new__">__new__</a>(S, ...) -> a new object with type S, a subtype of T</tt></dl>
78 Methods inherited from <a href="exceptions.html#BaseException">exceptions.BaseException</a>:<br>
79 <dl><dt><a name="CloudStorageError-__delattr__"><strong>__delattr__</strong></a>(...)</dt><dd><tt>x.<a href="#CloudStorageError-__delattr__">__delattr__</a>('name') <==> del x.name</tt></dd></dl>
81 <dl><dt><a name="CloudStorageError-__getattribute__"><strong>__getattribute__</strong></a>(...)</dt><dd><tt>x.<a href="#CloudStorageError-__getattribute__">__getattribute__</a>('name') <==> x.name</tt></dd></dl>
83 <dl><dt><a name="CloudStorageError-__getitem__"><strong>__getitem__</strong></a>(...)</dt><dd><tt>x.<a href="#CloudStorageError-__getitem__">__getitem__</a>(y) <==> x[y]</tt></dd></dl>
85 <dl><dt><a name="CloudStorageError-__getslice__"><strong>__getslice__</strong></a>(...)</dt><dd><tt>x.<a href="#CloudStorageError-__getslice__">__getslice__</a>(i, j) <==> x[i:j]<br>
87 Use of negative indices is not supported.</tt></dd></dl>
89 <dl><dt><a name="CloudStorageError-__reduce__"><strong>__reduce__</strong></a>(...)</dt></dl>
91 <dl><dt><a name="CloudStorageError-__repr__"><strong>__repr__</strong></a>(...)</dt><dd><tt>x.<a href="#CloudStorageError-__repr__">__repr__</a>() <==> repr(x)</tt></dd></dl>
93 <dl><dt><a name="CloudStorageError-__setattr__"><strong>__setattr__</strong></a>(...)</dt><dd><tt>x.<a href="#CloudStorageError-__setattr__">__setattr__</a>('name', value) <==> x.name = value</tt></dd></dl>
95 <dl><dt><a name="CloudStorageError-__setstate__"><strong>__setstate__</strong></a>(...)</dt></dl>
97 <dl><dt><a name="CloudStorageError-__str__"><strong>__str__</strong></a>(...)</dt><dd><tt>x.<a href="#CloudStorageError-__str__">__str__</a>() <==> str(x)</tt></dd></dl>
99 <dl><dt><a name="CloudStorageError-__unicode__"><strong>__unicode__</strong></a>(...)</dt></dl>
102 Data descriptors inherited from <a href="exceptions.html#BaseException">exceptions.BaseException</a>:<br>
103 <dl><dt><strong>__dict__</strong></dt>
105 <dl><dt><strong>args</strong></dt>
107 <dl><dt><strong>message</strong></dt>
109 </td></tr></table> <p>
110 <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section">
111 <tr bgcolor="#ffc8d8">
112 <td colspan=3 valign=bottom> <br>
113 <font color="#000000" face="helvetica, arial"><a name="CredentialsError">class <strong>CredentialsError</strong></a>(<a href="telemetry.page.cloud_storage.html#CloudStorageError">CloudStorageError</a>)</font></td></tr>
115 <tr><td bgcolor="#ffc8d8"><tt> </tt></td><td> </td>
116 <td width="100%"><dl><dt>Method resolution order:</dt>
117 <dd><a href="telemetry.page.cloud_storage.html#CredentialsError">CredentialsError</a></dd>
118 <dd><a href="telemetry.page.cloud_storage.html#CloudStorageError">CloudStorageError</a></dd>
119 <dd><a href="exceptions.html#Exception">exceptions.Exception</a></dd>
120 <dd><a href="exceptions.html#BaseException">exceptions.BaseException</a></dd>
121 <dd><a href="__builtin__.html#object">__builtin__.object</a></dd>
124 Methods defined here:<br>
125 <dl><dt><a name="CredentialsError-__init__"><strong>__init__</strong></a>(self, gsutil_path)</dt></dl>
128 Data descriptors inherited from <a href="telemetry.page.cloud_storage.html#CloudStorageError">CloudStorageError</a>:<br>
129 <dl><dt><strong>__weakref__</strong></dt>
130 <dd><tt>list of weak references to the object (if defined)</tt></dd>
133 Data and other attributes inherited from <a href="exceptions.html#Exception">exceptions.Exception</a>:<br>
134 <dl><dt><strong>__new__</strong> = <built-in method __new__ of type object><dd><tt>T.<a href="#CredentialsError-__new__">__new__</a>(S, ...) -> a new object with type S, a subtype of T</tt></dl>
137 Methods inherited from <a href="exceptions.html#BaseException">exceptions.BaseException</a>:<br>
138 <dl><dt><a name="CredentialsError-__delattr__"><strong>__delattr__</strong></a>(...)</dt><dd><tt>x.<a href="#CredentialsError-__delattr__">__delattr__</a>('name') <==> del x.name</tt></dd></dl>
140 <dl><dt><a name="CredentialsError-__getattribute__"><strong>__getattribute__</strong></a>(...)</dt><dd><tt>x.<a href="#CredentialsError-__getattribute__">__getattribute__</a>('name') <==> x.name</tt></dd></dl>
142 <dl><dt><a name="CredentialsError-__getitem__"><strong>__getitem__</strong></a>(...)</dt><dd><tt>x.<a href="#CredentialsError-__getitem__">__getitem__</a>(y) <==> x[y]</tt></dd></dl>
144 <dl><dt><a name="CredentialsError-__getslice__"><strong>__getslice__</strong></a>(...)</dt><dd><tt>x.<a href="#CredentialsError-__getslice__">__getslice__</a>(i, j) <==> x[i:j]<br>
146 Use of negative indices is not supported.</tt></dd></dl>
148 <dl><dt><a name="CredentialsError-__reduce__"><strong>__reduce__</strong></a>(...)</dt></dl>
150 <dl><dt><a name="CredentialsError-__repr__"><strong>__repr__</strong></a>(...)</dt><dd><tt>x.<a href="#CredentialsError-__repr__">__repr__</a>() <==> repr(x)</tt></dd></dl>
152 <dl><dt><a name="CredentialsError-__setattr__"><strong>__setattr__</strong></a>(...)</dt><dd><tt>x.<a href="#CredentialsError-__setattr__">__setattr__</a>('name', value) <==> x.name = value</tt></dd></dl>
154 <dl><dt><a name="CredentialsError-__setstate__"><strong>__setstate__</strong></a>(...)</dt></dl>
156 <dl><dt><a name="CredentialsError-__str__"><strong>__str__</strong></a>(...)</dt><dd><tt>x.<a href="#CredentialsError-__str__">__str__</a>() <==> str(x)</tt></dd></dl>
158 <dl><dt><a name="CredentialsError-__unicode__"><strong>__unicode__</strong></a>(...)</dt></dl>
161 Data descriptors inherited from <a href="exceptions.html#BaseException">exceptions.BaseException</a>:<br>
162 <dl><dt><strong>__dict__</strong></dt>
164 <dl><dt><strong>args</strong></dt>
166 <dl><dt><strong>message</strong></dt>
168 </td></tr></table> <p>
169 <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section">
170 <tr bgcolor="#ffc8d8">
171 <td colspan=3 valign=bottom> <br>
172 <font color="#000000" face="helvetica, arial"><a name="NotFoundError">class <strong>NotFoundError</strong></a>(<a href="telemetry.page.cloud_storage.html#CloudStorageError">CloudStorageError</a>)</font></td></tr>
174 <tr><td bgcolor="#ffc8d8"><tt> </tt></td><td> </td>
175 <td width="100%"><dl><dt>Method resolution order:</dt>
176 <dd><a href="telemetry.page.cloud_storage.html#NotFoundError">NotFoundError</a></dd>
177 <dd><a href="telemetry.page.cloud_storage.html#CloudStorageError">CloudStorageError</a></dd>
178 <dd><a href="exceptions.html#Exception">exceptions.Exception</a></dd>
179 <dd><a href="exceptions.html#BaseException">exceptions.BaseException</a></dd>
180 <dd><a href="__builtin__.html#object">__builtin__.object</a></dd>
183 Data descriptors inherited from <a href="telemetry.page.cloud_storage.html#CloudStorageError">CloudStorageError</a>:<br>
184 <dl><dt><strong>__weakref__</strong></dt>
185 <dd><tt>list of weak references to the object (if defined)</tt></dd>
188 Methods inherited from <a href="exceptions.html#Exception">exceptions.Exception</a>:<br>
189 <dl><dt><a name="NotFoundError-__init__"><strong>__init__</strong></a>(...)</dt><dd><tt>x.<a href="#NotFoundError-__init__">__init__</a>(...) initializes x; see help(type(x)) for signature</tt></dd></dl>
192 Data and other attributes inherited from <a href="exceptions.html#Exception">exceptions.Exception</a>:<br>
193 <dl><dt><strong>__new__</strong> = <built-in method __new__ of type object><dd><tt>T.<a href="#NotFoundError-__new__">__new__</a>(S, ...) -> a new object with type S, a subtype of T</tt></dl>
196 Methods inherited from <a href="exceptions.html#BaseException">exceptions.BaseException</a>:<br>
197 <dl><dt><a name="NotFoundError-__delattr__"><strong>__delattr__</strong></a>(...)</dt><dd><tt>x.<a href="#NotFoundError-__delattr__">__delattr__</a>('name') <==> del x.name</tt></dd></dl>
199 <dl><dt><a name="NotFoundError-__getattribute__"><strong>__getattribute__</strong></a>(...)</dt><dd><tt>x.<a href="#NotFoundError-__getattribute__">__getattribute__</a>('name') <==> x.name</tt></dd></dl>
201 <dl><dt><a name="NotFoundError-__getitem__"><strong>__getitem__</strong></a>(...)</dt><dd><tt>x.<a href="#NotFoundError-__getitem__">__getitem__</a>(y) <==> x[y]</tt></dd></dl>
203 <dl><dt><a name="NotFoundError-__getslice__"><strong>__getslice__</strong></a>(...)</dt><dd><tt>x.<a href="#NotFoundError-__getslice__">__getslice__</a>(i, j) <==> x[i:j]<br>
205 Use of negative indices is not supported.</tt></dd></dl>
207 <dl><dt><a name="NotFoundError-__reduce__"><strong>__reduce__</strong></a>(...)</dt></dl>
209 <dl><dt><a name="NotFoundError-__repr__"><strong>__repr__</strong></a>(...)</dt><dd><tt>x.<a href="#NotFoundError-__repr__">__repr__</a>() <==> repr(x)</tt></dd></dl>
211 <dl><dt><a name="NotFoundError-__setattr__"><strong>__setattr__</strong></a>(...)</dt><dd><tt>x.<a href="#NotFoundError-__setattr__">__setattr__</a>('name', value) <==> x.name = value</tt></dd></dl>
213 <dl><dt><a name="NotFoundError-__setstate__"><strong>__setstate__</strong></a>(...)</dt></dl>
215 <dl><dt><a name="NotFoundError-__str__"><strong>__str__</strong></a>(...)</dt><dd><tt>x.<a href="#NotFoundError-__str__">__str__</a>() <==> str(x)</tt></dd></dl>
217 <dl><dt><a name="NotFoundError-__unicode__"><strong>__unicode__</strong></a>(...)</dt></dl>
220 Data descriptors inherited from <a href="exceptions.html#BaseException">exceptions.BaseException</a>:<br>
221 <dl><dt><strong>__dict__</strong></dt>
223 <dl><dt><strong>args</strong></dt>
225 <dl><dt><strong>message</strong></dt>
227 </td></tr></table> <p>
228 <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section">
229 <tr bgcolor="#ffc8d8">
230 <td colspan=3 valign=bottom> <br>
231 <font color="#000000" face="helvetica, arial"><a name="PermissionError">class <strong>PermissionError</strong></a>(<a href="telemetry.page.cloud_storage.html#CloudStorageError">CloudStorageError</a>)</font></td></tr>
233 <tr><td bgcolor="#ffc8d8"><tt> </tt></td><td> </td>
234 <td width="100%"><dl><dt>Method resolution order:</dt>
235 <dd><a href="telemetry.page.cloud_storage.html#PermissionError">PermissionError</a></dd>
236 <dd><a href="telemetry.page.cloud_storage.html#CloudStorageError">CloudStorageError</a></dd>
237 <dd><a href="exceptions.html#Exception">exceptions.Exception</a></dd>
238 <dd><a href="exceptions.html#BaseException">exceptions.BaseException</a></dd>
239 <dd><a href="__builtin__.html#object">__builtin__.object</a></dd>
242 Methods defined here:<br>
243 <dl><dt><a name="PermissionError-__init__"><strong>__init__</strong></a>(self, gsutil_path)</dt></dl>
246 Data descriptors inherited from <a href="telemetry.page.cloud_storage.html#CloudStorageError">CloudStorageError</a>:<br>
247 <dl><dt><strong>__weakref__</strong></dt>
248 <dd><tt>list of weak references to the object (if defined)</tt></dd>
251 Data and other attributes inherited from <a href="exceptions.html#Exception">exceptions.Exception</a>:<br>
252 <dl><dt><strong>__new__</strong> = <built-in method __new__ of type object><dd><tt>T.<a href="#PermissionError-__new__">__new__</a>(S, ...) -> a new object with type S, a subtype of T</tt></dl>
255 Methods inherited from <a href="exceptions.html#BaseException">exceptions.BaseException</a>:<br>
256 <dl><dt><a name="PermissionError-__delattr__"><strong>__delattr__</strong></a>(...)</dt><dd><tt>x.<a href="#PermissionError-__delattr__">__delattr__</a>('name') <==> del x.name</tt></dd></dl>
258 <dl><dt><a name="PermissionError-__getattribute__"><strong>__getattribute__</strong></a>(...)</dt><dd><tt>x.<a href="#PermissionError-__getattribute__">__getattribute__</a>('name') <==> x.name</tt></dd></dl>
260 <dl><dt><a name="PermissionError-__getitem__"><strong>__getitem__</strong></a>(...)</dt><dd><tt>x.<a href="#PermissionError-__getitem__">__getitem__</a>(y) <==> x[y]</tt></dd></dl>
262 <dl><dt><a name="PermissionError-__getslice__"><strong>__getslice__</strong></a>(...)</dt><dd><tt>x.<a href="#PermissionError-__getslice__">__getslice__</a>(i, j) <==> x[i:j]<br>
264 Use of negative indices is not supported.</tt></dd></dl>
266 <dl><dt><a name="PermissionError-__reduce__"><strong>__reduce__</strong></a>(...)</dt></dl>
268 <dl><dt><a name="PermissionError-__repr__"><strong>__repr__</strong></a>(...)</dt><dd><tt>x.<a href="#PermissionError-__repr__">__repr__</a>() <==> repr(x)</tt></dd></dl>
270 <dl><dt><a name="PermissionError-__setattr__"><strong>__setattr__</strong></a>(...)</dt><dd><tt>x.<a href="#PermissionError-__setattr__">__setattr__</a>('name', value) <==> x.name = value</tt></dd></dl>
272 <dl><dt><a name="PermissionError-__setstate__"><strong>__setstate__</strong></a>(...)</dt></dl>
274 <dl><dt><a name="PermissionError-__str__"><strong>__str__</strong></a>(...)</dt><dd><tt>x.<a href="#PermissionError-__str__">__str__</a>() <==> str(x)</tt></dd></dl>
276 <dl><dt><a name="PermissionError-__unicode__"><strong>__unicode__</strong></a>(...)</dt></dl>
279 Data descriptors inherited from <a href="exceptions.html#BaseException">exceptions.BaseException</a>:<br>
280 <dl><dt><strong>__dict__</strong></dt>
282 <dl><dt><strong>args</strong></dt>
284 <dl><dt><strong>message</strong></dt>
286 </td></tr></table></td></tr></table><p>
287 <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section">
288 <tr bgcolor="#eeaa77">
289 <td colspan=3 valign=bottom> <br>
290 <font color="#ffffff" face="helvetica, arial"><big><strong>Functions</strong></big></font></td></tr>
292 <tr><td bgcolor="#eeaa77"><tt> </tt></td><td> </td>
293 <td width="100%"><dl><dt><a name="-Delete"><strong>Delete</strong></a>(bucket, remote_path)</dt></dl>
294 <dl><dt><a name="-Exists"><strong>Exists</strong></a>(bucket, remote_path)</dt></dl>
295 <dl><dt><a name="-FindGsutil"><strong>FindGsutil</strong></a>()</dt><dd><tt>Return the gsutil executable path. If we can't find it, download it.</tt></dd></dl>
296 <dl><dt><a name="-Get"><strong>Get</strong></a>(bucket, remote_path, local_path)</dt></dl>
297 <dl><dt><a name="-GetHash"><strong>GetHash</strong></a>(file_path)</dt><dd><tt>Calculates and returns the hash of the file at file_path.</tt></dd></dl>
298 <dl><dt><a name="-GetIfChanged"><strong>GetIfChanged</strong></a>(file_path, bucket<font color="#909090">=None</font>)</dt><dd><tt>Gets the file at file_path if it has a hash file that doesn't match.<br>
300 If the file is not in Cloud Storage, log a warning instead of raising an<br>
301 exception. We assume that the user just hasn't uploaded the file yet.<br>
304 True if the binary was changed.</tt></dd></dl>
305 <dl><dt><a name="-Insert"><strong>Insert</strong></a>(bucket, remote_path, local_path, publicly_readable<font color="#909090">=False</font>)</dt></dl>
306 <dl><dt><a name="-List"><strong>List</strong></a>(bucket)</dt></dl>
307 <dl><dt><a name="-Move"><strong>Move</strong></a>(bucket1, bucket2, remote_path)</dt></dl>
308 <dl><dt><a name="-SupportsProdaccess"><strong>SupportsProdaccess</strong></a>(gsutil_path)</dt></dl>
309 </td></tr></table><p>
310 <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section">
311 <tr bgcolor="#55aa55">
312 <td colspan=3 valign=bottom> <br>
313 <font color="#ffffff" face="helvetica, arial"><big><strong>Data</strong></big></font></td></tr>
315 <tr><td bgcolor="#55aa55"><tt> </tt></td><td> </td>
316 <td width="100%"><strong>INTERNAL_BUCKET</strong> = 'chrome-telemetry'<br>
317 <strong>PARTNER_BUCKET</strong> = 'chrome-partner-telemetry'<br>
318 <strong>PUBLIC_BUCKET</strong> = 'chromium-telemetry'</td></tr></table>