Troubleshooting Installation Via Pip

[Background: I am a product analyst with experience in python and would like to use the google cloud package for python to query directly from my company's BigQuery instead of using a bash wrapper and the gcloud sdk.]

I was unable to install use pip (I forget the exact error as this was yesterday) but it finally installed when I used the flag --ignore-installed

However, when I then ran from google.cloud import bigquery I then ran into an IOError: Permission Denied: <'path/to/protobuf-3.0.0-py2.7.egg/EGG-INFO/namespace_packages.txt'> which I solved by changing the permissions on that. Now, I receive this traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/google/cloud/bigquery/__init__.py", line 26, in <module>
    from google.cloud.bigquery.client import Client
  File "/Library/Python/2.7/site-packages/google/cloud/bigquery/client.py", line 18, in <module>
    from google.cloud.client import JSONClient
  File "/Library/Python/2.7/site-packages/google/cloud/client.py", line 20, in <module>
    from google.cloud._helpers import _determine_default_project
  File "/Library/Python/2.7/site-packages/google/cloud/_helpers.py", line 28, in <module>
    from google.protobuf import timestamp_pb2
  File "/Library/Python/2.7/site-packages/google/protobuf/timestamp_pb2.py", line 6, in <module>
    from google.protobuf import descriptor as _descriptor
  File "/Library/Python/2.7/site-packages/google/protobuf/descriptor.py", line 39, in <module>
    from google.protobuf.internal import api_implementation
ImportError: cannot import name api_implementation--

I'm not sure if this is an error with my installation or with the google cloud package itself, and I'm not sure exactly what the problem is, so apologies that I cannot ask a clearer question. Thank you.

@dhermes

@dhermes thank you

@emilyhoughkovacs

@emilyhoughkovacs Can you provide the full command you used to install? It looks like you have a bad google.protobuf install.

You may have b0rken site-packages, which is better to solve by pip uninstall (or just finding the content and deleting) than it is by fiddling with permissions.

From #2437 it seems like you installed from source. However, you don't need to / shouldn't do this. Instead you should just pip install google-cloud. In addition, I recommend you install into a virtualenv rather than using sudo.

@dhermes

@dhermes [sudo] pip install --upgrade google-cloud --ignore-installed

Interesting that it failed. Do a pip show google-cloud protobuf and paste the output? (Feel free to redact anything you like from the pasted output)

---
Metadata-Version: 1.1
Name: google-cloud
Version: 0.20.0.dev0
Summary: API Client library for Google Cloud
Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python
Author: Google Cloud Platform
Author-email: jjg+google-cloud-python@google.com
License: Apache 2.0
Location: /usr/local/lib/python2.7/site-packages/google_cloud-0.20.0.dev0-py2.7.egg
Requires: google-cloud-bigquery, google-cloud-bigtable, google-cloud-core, google-cloud-datastore, google-cloud-dns, google-cloud-language, google-cloud-logging, google-cloud-pubsub, google-cloud-storage
Classifiers:
  Development Status :: 4 - Beta
  Intended Audience :: Developers
  License :: OSI Approved :: Apache Software License
  Operating System :: OS Independent
  Programming Language :: Python :: 2
  Programming Language :: Python :: 2.7
  Programming Language :: Python :: 3
  Programming Language :: Python :: 3.4
  Programming Language :: Python :: 3.5
  Topic :: Internet
---
Metadata-Version: 1.1
Name: protobuf
Version: 3.0.0
Summary: Protocol Buffers
Home-page: https://developers.google.com/protocol-buffers/
Author: protobuf@googlegroups.com
Author-email: protobuf@googlegroups.com
License: New BSD License
Location: /Library/Python/2.7/site-packages/protobuf-3.0.0-py2.7.egg
Requires: six, setuptools
Classifiers:
  Programming Language :: Python
  Programming Language :: Python :: 2
  Programming Language :: Python :: 2.6
  Programming Language :: Python :: 2.7
  Programming Language :: Python :: 3
  Programming Language :: Python :: 3.3
  Programming Language :: Python :: 3.4

Note: my computer runs OS X 10.11.5 and I am not root (it is my work computer) so I had to run sudo -H pip show google-cloud protobuf for this to work. Not using sudo I received this traceback:

Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/site-packages/pip/commands/show.py", line 40, in run
    if not print_results(results, options.files):
  File "/usr/local/lib/python2.7/site-packages/pip/commands/show.py", line 125, in print_results
    for dist in distributions:
  File "/usr/local/lib/python2.7/site-packages/pip/commands/show.py", line 60, in search_packages_info
    'requires': [dep.project_name for dep in dist.requires()],
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2489, in requires
    dm = self._dep_map
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2474, in _dep_map
    for extra, reqs in split_sections(self._get_metadata(name)):
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2878, in split_sections
    for line in yield_lines(s):
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2174, in yield_lines
    for ss in strs:
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2503, in _get_metadata
    for line in self.get_metadata_lines(name):
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1477, in get_metadata_lines
    return yield_lines(self.get_metadata(name))
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1469, in get_metadata
    return self._get(self._fn(self.egg_info, name))
  File "/usr/local/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1580, in _get
    with open(path, 'rb') as stream:
IOError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/protobuf-3.0.0-py2.7.egg/EGG-INFO/requires.txt'```

The 0.20.0.dev0 shows you've installed from source, since 0.19.0 is the latest version on PyPI.

I recommend the following:

Uninstall these problematic packages:

sudo --set-home pip uninstall google-cloud protobuf

Install google-cloud from PyPI rather than from source:
```
sudo --set-home pip install google-cloud
```

Also, Mac OS X is essentially impossible to get a valid Python install because they have packages that you can't edit or delete (like six) that will be out of date.

I highly recommend you exclusively use virtualenvs on a Mac or just use a non-default Python like Anaconda or Homebrew.

Thank you! Yes, I was running into issues with six before so I definitely have been through the runaround. I will take your advice and first try conda and then a virtualenv.

Shall we close this out then?

👍

feat: add gapic-version generator option (#2440)

feat: remove redundant "started." messages from progress output (#2440)

chore: librarian release pull request: 20260207T015024Z (#2446)

Navigation Menu

Repository navigation

Troubleshooting Installation Via Pip #2440

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Troubleshooting Installation Via Pip #2440

Description

Activity

emilyhoughkovacs commented on Sep 27, 2016

dhermes commented on Sep 27, 2016

emilyhoughkovacs commented on Sep 27, 2016

dhermes commented on Sep 27, 2016

emilyhoughkovacs commented on Sep 27, 2016

dhermes commented on Sep 27, 2016

emilyhoughkovacs commented on Sep 27, 2016

dhermes commented on Sep 27, 2016

emilyhoughkovacs commented on Sep 28, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions